tencentyun / cos-java-sdk-v5 Goto Github PK
View Code? Open in Web Editor NEWjava sdk for qcloud cos v5 (xml api)
License: MIT License
java sdk for qcloud cos v5 (xml api)
License: MIT License
我们发现了5.6.36版本在maven中发布了,但是github中并没有相应的tag;
请问tag缺失的原因是什么?(发现提交记录里有相关版本发布的readme)
2) 针对该问题,我们提出了一个方法,能够找到version 可能对应的真实commit, 以帮助开发者快速定位问题代码,完成漏洞修复。 以下是我们方法给出的可能的真实commits,请您帮忙确认下是否包含了真实的commit,如果不是,应该是哪一个?
['a01c54fcd40da235eac305a6e8d2fd3d08859d9b', 'a5d349483b95859f1b0ba7777aba4bcf424cfb5c', '97249264e5a36811ce4cd74ef22c3badc6a6b37f', '6ff5f913c7ee4a95c2083b0abcf7625cc2847733']
请问如何支持MultipartFile类型文件
把HeadObjectRequest加上去吧,需要检查一下是否已经存在,防止重复上传
调用 deleteObjects 删除 object 无效,但是 deleteObject 是成功的
EndPointSuffix 设置为 cos.ap-beijing.myqcloud.com, listBuckets 接口报错
报错信息
com.qcloud.cos.exception.CosServiceException: Missing required header for this request: Appid (Status Code: 400; Error Code: InvalidRequest; Request ID: NWI3YzJlYjdfOGMyODVkNjRfYWZjMl9mZWZiN2E=); Trace ID: OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODczNTBmNjMwZmQ0MTZkMjg0NjlkNTYyNmY4ZTRkZTk0NzgyODEyZmZkMzFkOTczNGZhZTI1N2FiZmI0M2FiNzBmYTQ5MzFmODcxMjlkN2RjMGI3ZjU1NWQ2NjkxMDE3N2M=
出错原因: 请求地址错误
service.cos.ap-beijing.myqcloud.com
您好,我想问下在5.4.4最新的版本能用EndPoint吗?
我在ClientConfig 这个类里找到里,但是endPointSuffix 是null。所以我想了解下具体的用法??
@wucheng
如果prefix以"/"开头,则无法返回结果,需要手工把“/"去掉
SpringBoot 2.4.2
cos_api 5.6.105
@RestController
public class WebController {
@GetMapping("/download")
public void downloadFile(HttpServletResponse response) {
COSUtil.downloadFileByInputStream("img/flower.png", response);
}
}
@Slf4j
public class COSUtil {
private static final String SECRET_ID = "XXX";
private static final String SECRET_KEY = "XXX";
private static final String APPID = "XXX";
private static final String BUCKET_NAME = "XXX-" + APPID;
private static final String LOCATION_NAME = "XXX";
public void downloadFileByInputStream(String remotePath, HttpServletResponse response) {
COSCredentials credentials = new BasicCOSCredentials(SECRET_ID, SECRET_KEY);
ClientConfig clientConfig = new ClientConfig(new Region(LOCATION_NAME));
COSClient cosClient = new COSClient(credentials, clientConfig);
GetObjectRequest getObjectRequest = new GetObjectRequest(BUCKET_NAME, remotePath);
COSObject cosObject = cosClient.getObject(getObjectRequest);
try (COSObjectInputStream cosObjectInput = cosObject.getObjectContent()) {
byte[] buffer = new byte[cosObjectInput.available()];
ServletOutputStream outputStream = response.getOutputStream();
int read = cosObjectInput.read(buffer);
while (read != -1) {
outputStream.write(buffer);
outputStream.flush();
read = cosObjectInput.read(buffer);
}
} catch (IOException e) {
log.error(e.getMessage());
}
cosClient.shutdown();
}
}
调用COSClient对象的generatePresignedUrl方法生成预签名链接,用于上传、下载文件,该链接目前固定以http://协议头开始,希望可以优化,根据Origin头自动识别协议,或其他方式参数传入决定。
COSSigner cosSigner = new COSSigner(); String authStr = cosSigner.buildAuthorizationStr(request.getHttpMethod(), request.getResourcePath(), request.getHeaders(), request.getParameters(), cred, req.getExpiration()); StringBuilder strBuilder = new StringBuilder(); strBuilder.append("http://").append(formatBucket(bucketName, cred.getCOSAppId()));
再次打扰:
对于crc64功能上线以前的文件不能返回crc64值
”,我在官网没看到这个功能是什么时候上线的,方便在这告知一下这个时间点吗?我们公司用cos已经蛮多年了,需要确认一下这个上线时间。 public static String getCRC64FromFile(File file) throws IOException {
final CRC64 total = new CRC64();
try (FileInputStream stream = new FileInputStream(file)) {
final byte[] b = new byte[1024 * 1024];
while (true) {
final int read = stream.read(b);
if (read <= 0) {
break;
}
total.update(b, read);
}
}
return String.valueOf(total.getValue());
}
COSSigner类本身具有过期时间属性private long signExpiredTime,在调用buildAuthorizationStr方法时,还需使用Date型参数,设计冗余,调用起来麻烦,且通过属性设置的过期时间和方法里的过期时间不知道以哪个为准。V5.1.4版本的buildAuthorizationStr方法的参数设计已经可以满足需求。在升级其他功能时发现方法调用改变得比较难用。
都没有和数据万象相结合的接口吗?那个数据万象也没有相应的sdk...
public static void SimpleUploadFileFromStream() {
....
String key = "aaa/bbb.jpg";
//这里加载文件
File localFile = new File("src/test/resources/len10M.txt");
//这里创建流,但未见读取文件代码
InputStream input = new ByteArrayInputStream(new byte[10]);
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.setContentLength(10);
objectMetadata.setContentType("image/jpeg");
//这里直接传入了input
PutObjectRequest putObjectRequest =new PutObjectRequest(bucketName, key, input, objectMetadata);
putObjectRequest.setStorageClass(StorageClass.Standard_IA);
....
}
版本5.5.1
调用其中的跨园区拷贝代码,TransferManagerDemo.java中的copyFileForDiffRegion方法。
报错显示:
com.qcloud.cos.exception.CosServiceException: Not Found (Status Code: 404; Error Code: 404 Not Found; Request ID: NWNhNDUzNGFfOTcxYzBiMDlfNjNhZl84YjVlMmY=); Trace ID: OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODc0OWRkZjk0ZDM1NmI1M2E2MTRlY2MzZDhmNmI5MWI1OTBjYzE2MjAxN2M1MzJiOTdkZjMxMDVlYTZjN2FiMmI0NTI1ZGFjMGQ1ZjQwZmYyN2NjODNmNzJiYzQwMmUyYTY=
at com.qcloud.cos.http.DefaultCosHttpClient.handlerErrorMessage(DefaultCosHttpClient.java:302)
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:402)
at com.qcloud.cos.COSClient.invoke(COSClient.java:524)
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1088)
at com.qcloud.cos.transfer.TransferManager.copy(TransferManager.java:1399)
at com.xiongjie.Application.haha(Application.java:722)
at com.xiongjie.Application.main(Application.java:67)
检测到存在待处理的应用漏洞:FasterXML jackson-databind 代码问题漏洞(CVE-2022-42003),威胁等级:高危
ImageAuditingResponse没有result返回
为什么 cosclient.setObjectAcl(ClientConfig.BUCKET_NAME, "123.m4a", CannedAccessControlList.Private)
设置为私有读写,浏览器还是能访问.... 这个有什么作用?
没有删除文件夹功能,貌似list某个文件夹下的文件和目录的功能都没
运行一段时候后,发现请求no response,查看日志发现所有请求提示
Timeout waiting for connection from pool,使用的是默认的1000连接配置
Provide callback style or CompletableFuture
base api
程序中未使用单例模式,每次连接都重新new一个COSClient对象,jvm占用内存一直升高,不会自动回收,直到出现OOM。怀疑内部存在引用未释放。
修改COSClient为单例模式调用之后,经过压测,内存基本保持稳定。
但使用临时密钥初始化时,需要将临时密钥初始化进去,因此需要有这种应用场景,多次创建对象的情况。
希望上传完成后,返回值当中加入完整的访问路径
在spark任务结束时会报如下错误:
10:40:22[dispatcher-event-loop-0] INFO org.apache.spark.scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint.logInfo:57 - OutputCommitCoordinator stopped!
10:40:22[Thread-1] INFO org.apache.spark.SparkContext.logInfo:57 - Successfully stopped SparkContext
10:40:22[Thread-1] INFO org.apache.spark.util.ShutdownHookManager.logInfo:57 - Shutdown hook called
10:40:22[Thread-1] INFO org.apache.spark.util.ShutdownHookManager.logInfo:57 - Deleting directory /private/var/folders/9d/qtc20f6x197431jvthgs58zr0000gn/T/spark-f52383f9-6554-42f5-8296-ac65779a77e7
10:40:22[Thread-1] INFO org.apache.hadoop.fs.BufferPool.close:265 - Close a buffer pool instance.
10:40:22[Thread-1] INFO org.apache.hadoop.fs.BufferPool.close:278 - Begin to release the buffers.
10:40:22[Thread-1] ERROR com.qcloud.cos.http.DefaultCosHttpClient.exeute:498 - httpClient execute occur a unknow exception, httpRequest: endpoint: dlf-test-1258469122.cos.ap-beijing.myqcloud.com, resourcepath: /tmp/hive/MjUxMDExMzQ5OkRMRjp6WUJFbUpGZzp1N0N6dWpG/b57883f4-d810-49ed-a920-36e1c7cf92e1, httpMethod: HEAD, headers { Authorization : q-sign-algorithm=sha1&q-ak=AKIDDCvEI6MxqUt2ydvE7jQpwta1B2gS9DF3&q-sign-time=1622428822;1622432422&q-key-time=1622428822;1622432422&q-header-list=host&q-url-param-list=&q-signature=a94b5c55ca67f69330f51ab5efb5321b3992a89e, User-Agent : cos-hadoop-plugin-v5.9.4, Host : dlf-test-1258469122.cos.ap-beijing.myqcloud.com, }, params: { }
java.lang.IllegalStateException: Connection pool shut down
at com.qcloud.cos.thirdparty.org.apache.http.util.Asserts.check(Asserts.java:34) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:189) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:257) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:409) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:471) [cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.COSClient.invoke(COSClient.java:653) [cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1271) [cos_api-bundle-5.6.42.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.callCOSClientWithRetry(CosNativeFileSystemStore.java:1205) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.queryObjectMetadata(CosNativeFileSystemStore.java:471) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.retrieveMetadata(CosNativeFileSystemStore.java:544) [hadoop-cos-2.7.5-5.9.4.jar:?]
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) [hadoop-common-2.7.4.jar:?]
at com.sun.proxy.$Proxy46.retrieveMetadata(Unknown Source) [?:?]
at org.apache.hadoop.fs.CosFileSystem.getFileStatus(CosFileSystem.java:428) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem.processDeleteOnExit(FileSystem.java:1409) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:2070) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.CosFileSystem.close(CosFileSystem.java:1160) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2760) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2777) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) [hadoop-common-2.7.4.jar:?]
javascript上传 java生成签名的demo好像没有 只有php和node.js的吗?
cos-java-sdk作为一个api,希望不要直接引入slf4j-log4j12,改成引入slf4j-api
version:5.6.38
java: 1.8.0_191
目前的解决方法是在添加metadata时,手动对value值再进行一次urlencode,但是这样在控制台看到的信息是编码后的。
在COSSigner里对headers已经进行了urlencode,不明白为什么还是会报签名失败
http正常使用,https报错如下图
代理配置如下
Established tunneling session to airbyte-oss-test-1258678410.cos.ap-nanjing.myqcloud.com:443. Port forwarding started on /127.0.0.1:42685
使用时配置了
clientConfig.httpProxyIp = "127.0.0.1"
clientConfig.httpProxyPort = port
手动注入为hppts后报错信息变更为
com.qcloud.cos.exception.CosClientException: Certificate for <127.0.0.1> doesn't match any of the subject alternative names: [*.cos.ap-nanjing.myqcloud.com, *.cn-east.myqcloud.com, *.cn-north.myqcloud.com, *.cn-south-2.myqcloud.com, *.cn-south.myqcloud.com, *.cn-southwest.myqcloud.com, *.cos-cdn.ap-nanjing.myqcloud.com, *.cos-cdn.ap-nanjing.tencentcos.cn, *.cos-cdn.ap-nanjing.tencentcos.com, *.cos-cdn.ap-nanjing.tencentcos.com.cn, *.cos-control.ap-nanjing.myqcloud.com, *.cos-control.ap-nanjing.tencentcos.cn, *.cos-control.ap-nanjing.tencentcos.com, *.cos-control.ap-nanjing.tencentcos.com.cn, *.cos-ext.ap-nanjing.myqcloud.com, *.cos-ext.ap-nanjing.tencentcos.com, *.cos-internal.accelerate.tencentcos.cn, *.cos-internal.accelerate.tencentcos.com.cn, *.cos-internal.ap-nanjing.myqcloud.com, *.cos-internal.ap-nanjing.tencentcos.cn, *.cos-internal.ap-nanjing.tencentcos.com, *.cos-internal.ap-nanjing.tencentcos.com.cn, *.cos.accelerate.myqcloud.com, *.cos.accelerate.tencentcos.cn, *.cos.accelerate.tencentcos.com, *.cos.accelerate.tencentcos.com.cn, *.cos.ap-nanjing.tencentcos.cn, *.cos.ap-nanjing.tencentcos.com, *.cos.ap-nanjing.tencentcos.com.cn, *.file.myqcloud.com, *.file.tencentcos.cn, *.file.tencentcos.com.cn, cos-cdn.ap-nanjing.myqcloud.com, cos-cdn.ap-nanjing.tencentcos.cn, cos-cdn.ap-nanjing.tencentcos.com, cos-cdn.ap-nanjing.tencentcos.com.cn, cos-control.ap-nanjing.myqcloud.com, cos-control.ap-nanjing.tencentcos.cn, cos-control.ap-nanjing.tencentcos.com, cos-control.ap-nanjing.tencentcos.com.cn, cos-ext.ap-nanjing.myqcloud.com, cos-ext.ap-nanjing.tencentcos.com, cos-internal.accelerate.tencentcos.cn, cos-internal.accelerate.tencentcos.com.cn, cos-internal.ap-nanjing.myqcloud.com, cos-internal.ap-nanjing.tencentcos.cn, cos-internal.ap-nanjing.tencentcos.com, cos-internal.ap-nanjing.tencentcos.com.cn, cos.accelerate.myqcloud.com, cos.accelerate.tencentcos.cn, cos.accelerate.tencentcos.com, cos.accelerate.tencentcos.com.cn, cos.ap-nanjing.tencentcos.cn, cos.ap-nanjing.tencentcos.com, cos.ap-nanjing.tencentcos.com.cn, file.myqcloud.com, file.tencentcos.cn, file.tencentcos.com.cn, cos.ap-nanjing.myqcloud.com]
at com.qcloud.cos.utils.ExceptionUtils.createClientException(ExceptionUtils.java:47) ~[cos_api-5.6.153.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.executeRequest(DefaultCosHttpClient.java:589) ~[cos_api-5.6.153.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:483) ~[cos_api-5.6.153.jar:?]
at com.qcloud.cos.COSClient.invoke(COSClient.java:614) ~[cos_api-5.6.153.jar:?]
at com.qcloud.cos.COSClient.listObjects(COSClient.java:1961) ~[cos_api-5.6.153.jar:?]
拷贝方法执行完未报错,就说明文件已经拷贝过去了吗?
此时我马上去下载新产生的文件,可以保证文件一定正确不?
我copy完以后去调取文件元数据的getContentMD5 发现是空的
version: 5.5.5
org.apache.http.client.ClientProtocolException: null
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:369)
at com.qcloud.cos.COSClient.invoke(COSClient.java:530)
at com.qcloud.cos.COSClient.putObject(COSClient.java:848)
at com.innjoy.pms.middleware.util.UploadFileUtil.upload(UploadFileUtil.java:61)
at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.uploadImg(IdentificationMigrationServiceImpl.java:148)
at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.getImgUrl(IdentificationMigrationServiceImpl.java:107)
at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.saveIdentification(IdentificationMigrationServiceImpl.java:67)
at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl$$FastClassBySpringCGLIB$$810649c.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:108)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
... 22 common frames omitted
Caused by: java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
at org.apache.http.impl.io.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:122)
at org.apache.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:179)
at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:134)
at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121)
at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
... 24 common frames omitted
是每次调用完一个方法都要执行一次shutdown吗?
我目前是这样:
COSClient cosclient = new COSClient(cred, clientConfig);
try {
cosclient.upload(...);
cosclient.upload(...);
cosclient.upload(...);
...
} finally {
cosclient.shutdown();
}
是否合理。
使用场景:
后端生成 authorizationStr,前端利用这个 authorizationStr 操作 cos 资源。
目前sdk提供:
COSSigner.buildAuthorizationStr() 可以用来生成 authorizationStr,但是直接用到 URL 的 sign 参数中会出现Access denies。只能放在 HTTP 请求的 Authorization header才能有权访问资源。
要想使用在 URL 加 sign 参数的方式来用这个 authorizationStr,必须要做一下以下操作:
UrlEncoderUtils.encode(authorizationStr)
那么,能不能提供一个现成的 API 来生成这个可以用在 URL 上的授权参数呢?
比如在 COSSigner 中加上以下方法:
public String sign(HttpMethodName methodName, String resouce_path,
COSCredentials cred, Date expiredTime) {
String authorizationStr = buildAuthorizationStr(methodName, resource_path, cred, expiredTime);
return UrlEncoderUtils.encode(authorizationStr);
}
06-09 18:10:08.843
文件是通过 spark 直接导入到 cosn://buc-13424123/key1/da/pp1/part-00000-f124f8f4-5b44-4c83-aece-6cd20629860b-c000.snappy.parquet
获取到的 ETag 为:001a387d4ca31d1952d3409c63503c71-3
得到的 ETag 位数超过32位,后面的 “-3” 去掉也不是正确的校验值。
使用sdk进行分片上传时,第一个分片能上传,
第二个分片上传时就报超时错误。
普通上传没有问题。
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:426)
at com.qcloud.cos.http.DefaultCosHttpClient.executeRequest(DefaultCosHttpClient.java:586)
执行命令
mvn dependency:tree -Dverbose
出现了okio:jar:1.6.0:compile - omitted for conflict with 1.12.0
[INFO] ------------------------------------------------------------------------
[INFO] Building cos-java-sdk 5.6.42
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ cos_api ---
[INFO] com.qcloud:cos_api:jar:5.6.42
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
[INFO] | \- commons-logging:commons-logging:jar:1.2:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.26:compile
[INFO] +- commons-codec:commons-codec:jar:1.10:compile
[INFO] +- joda-time:joda-time:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.1:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.1:compile
[INFO] | \- com.fasterxml.jackson.core:jackson-core:jar:2.12.1:compile
[INFO] +- com.tencentcloudapi:tencentcloud-sdk-java-kms:jar:3.1.213:compile
[INFO] | \- com.tencentcloudapi:tencentcloud-sdk-java-common:jar:3.1.213:compile
[INFO] | +- (commons-logging:commons-logging:jar:1.2:compile - omitted for duplicate)
[INFO] | +- com.squareup.okhttp:logging-interceptor:jar:2.7.5:compile
[INFO] | | \- com.squareup.okhttp:okhttp:jar:2.7.5:compile
[INFO] | | \- (com.squareup.okio:okio:jar:1.6.0:compile - omitted for conflict with 1.12.0)
[INFO] | +- com.squareup.okio:okio:jar:1.12.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] | \- javax.xml.bind:jaxb-api:jar:2.3.0:compile
[INFO] +- junit:junit:jar:4.13.1:test
[INFO] | \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.64:compile
[INFO] +- com.qcloud:qcloud-java-sdk:jar:2.0.6:test
[INFO] \- com.tencent.cloud:cos-sts-java:jar:3.0.8:test
[INFO] \- org.json:json:jar:20180130:test
请问,可否实例化一个COSClient,然后多个线程使用这一个实例进行COS文件操作
新的文档中持久化处理一节中,说:
您只需要在请求包头部中加入 Pic-Operations 项并设置好相应参数,就可在图片上传时实现相应的图片处理,并可将原图和处理结果存入到 COS。
图片上传的请求包与 COS V5 简单上传文件接口一致,只在请求包头部增加图片处理参数。
也提到了,你处理的后的结果是通过 http Response 的 content 中的 UploadResult 节点返回:
<UploadResult>
<OriginalInfo>
...
</OriginalInfo>
<ProcessResults>
...
</ProcessResults>
</UploadResult>
如果直接使用 CosClient 的 putObject 上传文件,经过 CosMetadataResponseHandler 处理,Response 的 content 就丢弃了,也就是说,无法通过 putObject 获取 Pic-Operations 的处理结果,只能自己全部实现整个上传过程。这个问题能修复下么?
25升级到28后,运行一天左右.就会大面积报下面的错误
com.qcloud.cos.exception.CosClientException: httpClient execute occur a unknow exception, httpRequest: endpoint: xxx-123456.cos.ap-shanghai.myqcloud.com, resourcepath: /projects/A__2020-03/project_5e67509360b2bcd915725ae9/task_5f27742a60b223ab27ac5b42/, httpMethod: HEAD, headers { Authorization : q-sign-algorithm=sha1&q-ak=sssss&q-sign-time=1596589498;1596593098&q-key-time=1596589498;1596593098&q-header-list=host&q-url-param-list=&q-signature=4d66cb9db906caa2f4c4711bb624d0afb6dc7513, User-Agent : cos-java-sdk-v5.6.28/Linux/jdk-1.8.0_151/Java HotSpot(TM) 64-Bit Server VM, Host : xxx-123456.cos.ap-shanghai.myqcloud.com, }, params: { }
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:501)
at com.qcloud.cos.COSClient.invoke(COSClient.java:601)
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1211)
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1188)
at com.qcloud.cos.COSClient.doesObjectExist(COSClient.java:1175)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Connection pool shut down
at org.apache.http.util.Asserts.check(Asserts.java:34)
at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:251)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:175)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:414)
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:475)
... 22 more
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.