GithubHelp home page GithubHelp logo

cos-java-sdk-v5's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cos-java-sdk-v5's Issues

5.6.36 tag缺失问题

我们发现了5.6.36版本在maven中发布了,但是github中并没有相应的tag;

请问tag缺失的原因是什么?(发现提交记录里有相关版本发布的readme)

  • a、该版本存在代码问题,如存在漏洞、兼容性问题、界面方面问题等。
  • b、该版本出现功能性问题,如功能规划不合理、基础功能不能使用等。
  • c、忘记打tag
  • d、不在意或者嫌操作麻烦
  • e、其他原因,_

2) 针对该问题,我们提出了一个方法,能够找到version 可能对应的真实commit, 以帮助开发者快速定位问题代码,完成漏洞修复。 以下是我们方法给出的可能的真实commits,请您帮忙确认下是否包含了真实的commit,如果不是,应该是哪一个?

['a01c54fcd40da235eac305a6e8d2fd3d08859d9b', 'a5d349483b95859f1b0ba7777aba4bcf424cfb5c', '97249264e5a36811ce4cd74ef22c3badc6a6b37f', '6ff5f913c7ee4a95c2083b0abcf7625cc2847733']

设置EndPointSuffix后,listBuckets 接口报错

EndPointSuffix 设置为 cos.ap-beijing.myqcloud.com, listBuckets 接口报错

报错信息

com.qcloud.cos.exception.CosServiceException: Missing required header for this request: Appid (Status Code: 400; Error Code: InvalidRequest; Request ID: NWI3YzJlYjdfOGMyODVkNjRfYWZjMl9mZWZiN2E=); Trace ID: OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODczNTBmNjMwZmQ0MTZkMjg0NjlkNTYyNmY4ZTRkZTk0NzgyODEyZmZkMzFkOTczNGZhZTI1N2FiZmI0M2FiNzBmYTQ5MzFmODcxMjlkN2RjMGI3ZjU1NWQ2NjkxMDE3N2M=

出错原因: 请求地址错误

service.cos.ap-beijing.myqcloud.com

endPointSuffix 问题

您好,我想问下在5.4.4最新的版本能用EndPoint吗?
我在ClientConfig 这个类里找到里,但是endPointSuffix 是null。所以我想了解下具体的用法??
@wucheng

从下载输出流COSObjectInputStream获取的图片数据错误

依赖

SpringBoot 2.4.2
cos_api 5.6.105

WebController

@RestController
public class WebController {
    @GetMapping("/download")
    public void downloadFile(HttpServletResponse response) {
        COSUtil.downloadFileByInputStream("img/flower.png", response);
    }
}

COSUtil

@Slf4j
public class COSUtil {
    private static final String SECRET_ID = "XXX";
    private static final String SECRET_KEY = "XXX";
    private static final String APPID = "XXX";
    private static final String BUCKET_NAME = "XXX-" + APPID;
    private static final String LOCATION_NAME = "XXX";
    
    public void downloadFileByInputStream(String remotePath, HttpServletResponse response) {
        COSCredentials credentials = new BasicCOSCredentials(SECRET_ID, SECRET_KEY);
        ClientConfig clientConfig = new ClientConfig(new Region(LOCATION_NAME));
        COSClient cosClient = new COSClient(credentials, clientConfig);
        GetObjectRequest getObjectRequest = new GetObjectRequest(BUCKET_NAME, remotePath);
        COSObject cosObject = cosClient.getObject(getObjectRequest);
        try (COSObjectInputStream cosObjectInput = cosObject.getObjectContent()) {
            byte[] buffer = new byte[cosObjectInput.available()];
            ServletOutputStream outputStream = response.getOutputStream();
            int read = cosObjectInput.read(buffer);
            while (read != -1) {
                outputStream.write(buffer);
                outputStream.flush();
                read = cosObjectInput.read(buffer);
            }
        } catch (IOException e) {
            log.error(e.getMessage());
        }
        cosClient.shutdown();
    }
}

原图

flower

浏览器访问图片

browser

生成预签名链接generatePresignedUrl固定返回http协议头,希望可以根据Origin进行区分。

调用COSClient对象的generatePresignedUrl方法生成预签名链接,用于上传、下载文件,该链接目前固定以http://协议头开始,希望可以优化,根据Origin头自动识别协议,或其他方式参数传入决定。
COSSigner cosSigner = new COSSigner(); String authStr = cosSigner.buildAuthorizationStr(request.getHttpMethod(), request.getResourcePath(), request.getHeaders(), request.getParameters(), cred, req.getExpiration()); StringBuilder strBuilder = new StringBuilder(); strBuilder.append("http://").append(formatBucket(bucketName, cred.getCOSAppId()));

关于计算本地文件的CRC64

再次打扰:

  1. 使用sdk内置的CRC64类计算出来的值始终不对,请问是不是我哪个地方有错?
  2. 对于crc64功能上线以前的文件不能返回crc64值”,我在官网没看到这个功能是什么时候上线的,方便在这告知一下这个时间点吗?我们公司用cos已经蛮多年了,需要确认一下这个上线时间。
   public static String getCRC64FromFile(File file) throws IOException {
        final CRC64 total = new CRC64();
        try (FileInputStream stream = new FileInputStream(file)) {
            final byte[] b = new byte[1024 * 1024];
            while (true) {
                final int read = stream.read(b);
                if (read <= 0) {
                    break;
                }
                total.update(b, read);
            }
        }
        return String.valueOf(total.getValue());
    }

V5.2.4-COSSigner类buildAuthorizationStr方法参数设计不合理

COSSigner类本身具有过期时间属性private long signExpiredTime,在调用buildAuthorizationStr方法时,还需使用Date型参数,设计冗余,调用起来麻烦,且通过属性设置的过期时间和方法里的过期时间不知道以哪个为准。V5.1.4版本的buildAuthorizationStr方法的参数设计已经可以满足需求。在升级其他功能时发现方法调用改变得比较难用。

通过文件流上传对象Demo 代码不完善

    
    public static void SimpleUploadFileFromStream() {
      ....
        String key = "aaa/bbb.jpg";
        //这里加载文件
        File localFile = new File("src/test/resources/len10M.txt");
        //这里创建流,但未见读取文件代码
        InputStream input = new ByteArrayInputStream(new byte[10]);
        ObjectMetadata objectMetadata = new ObjectMetadata();
       
        objectMetadata.setContentLength(10);
       
        objectMetadata.setContentType("image/jpeg");
        //这里直接传入了input
        PutObjectRequest putObjectRequest =new PutObjectRequest(bucketName, key, input, objectMetadata);
        
        putObjectRequest.setStorageClass(StorageClass.Standard_IA);
      ....
  }

跨园区拷贝, 即将一个园区的文件拷贝到另一个园区:失败

版本5.5.1
调用其中的跨园区拷贝代码,TransferManagerDemo.java中的copyFileForDiffRegion方法。
报错显示:
com.qcloud.cos.exception.CosServiceException: Not Found (Status Code: 404; Error Code: 404 Not Found; Request ID: NWNhNDUzNGFfOTcxYzBiMDlfNjNhZl84YjVlMmY=); Trace ID: OGVmYzZiMmQzYjA2OWNhODk0NTRkMTBiOWVmMDAxODc0OWRkZjk0ZDM1NmI1M2E2MTRlY2MzZDhmNmI5MWI1OTBjYzE2MjAxN2M1MzJiOTdkZjMxMDVlYTZjN2FiMmI0NTI1ZGFjMGQ1ZjQwZmYyN2NjODNmNzJiYzQwMmUyYTY=
at com.qcloud.cos.http.DefaultCosHttpClient.handlerErrorMessage(DefaultCosHttpClient.java:302)
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:402)
at com.qcloud.cos.COSClient.invoke(COSClient.java:524)
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1088)
at com.qcloud.cos.transfer.TransferManager.copy(TransferManager.java:1399)
at com.xiongjie.Application.haha(Application.java:722)
at com.xiongjie.Application.main(Application.java:67)

acl设置了无效 没有删除文件夹功能

为什么 cosclient.setObjectAcl(ClientConfig.BUCKET_NAME, "123.m4a", CannedAccessControlList.Private)
设置为私有读写,浏览器还是能访问.... 这个有什么作用?
没有删除文件夹功能,貌似list某个文件夹下的文件和目录的功能都没

COSClient多次实例化出现内存泄漏,无法回收内存

程序中未使用单例模式,每次连接都重新new一个COSClient对象,jvm占用内存一直升高,不会自动回收,直到出现OOM。怀疑内部存在引用未释放。
修改COSClient为单例模式调用之后,经过压测,内存基本保持稳定。
但使用临时密钥初始化时,需要将临时密钥初始化进去,因此需要有这种应用场景,多次创建对象的情况。

spark操作hive表任务结束时报错

image
在spark任务结束时会报如下错误:
10:40:22[dispatcher-event-loop-0] INFO org.apache.spark.scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint.logInfo:57 - OutputCommitCoordinator stopped!
10:40:22[Thread-1] INFO org.apache.spark.SparkContext.logInfo:57 - Successfully stopped SparkContext
10:40:22[Thread-1] INFO org.apache.spark.util.ShutdownHookManager.logInfo:57 - Shutdown hook called
10:40:22[Thread-1] INFO org.apache.spark.util.ShutdownHookManager.logInfo:57 - Deleting directory /private/var/folders/9d/qtc20f6x197431jvthgs58zr0000gn/T/spark-f52383f9-6554-42f5-8296-ac65779a77e7
10:40:22[Thread-1] INFO org.apache.hadoop.fs.BufferPool.close:265 - Close a buffer pool instance.
10:40:22[Thread-1] INFO org.apache.hadoop.fs.BufferPool.close:278 - Begin to release the buffers.
10:40:22[Thread-1] ERROR com.qcloud.cos.http.DefaultCosHttpClient.exeute:498 - httpClient execute occur a unknow exception, httpRequest: endpoint: dlf-test-1258469122.cos.ap-beijing.myqcloud.com, resourcepath: /tmp/hive/MjUxMDExMzQ5OkRMRjp6WUJFbUpGZzp1N0N6dWpG/b57883f4-d810-49ed-a920-36e1c7cf92e1, httpMethod: HEAD, headers { Authorization : q-sign-algorithm=sha1&q-ak=AKIDDCvEI6MxqUt2ydvE7jQpwta1B2gS9DF3&q-sign-time=1622428822;1622432422&q-key-time=1622428822;1622432422&q-header-list=host&q-url-param-list=&q-signature=a94b5c55ca67f69330f51ab5efb5321b3992a89e, User-Agent : cos-hadoop-plugin-v5.9.4, Host : dlf-test-1258469122.cos.ap-beijing.myqcloud.com, }, params: { }
java.lang.IllegalStateException: Connection pool shut down
at com.qcloud.cos.thirdparty.org.apache.http.util.Asserts.check(Asserts.java:34) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:189) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:257) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.thirdparty.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:409) ~[cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:471) [cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.COSClient.invoke(COSClient.java:653) [cos_api-bundle-5.6.42.jar:?]
at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1271) [cos_api-bundle-5.6.42.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.callCOSClientWithRetry(CosNativeFileSystemStore.java:1205) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.queryObjectMetadata(CosNativeFileSystemStore.java:471) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.CosNativeFileSystemStore.retrieveMetadata(CosNativeFileSystemStore.java:544) [hadoop-cos-2.7.5-5.9.4.jar:?]
at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_251]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_251]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) [hadoop-common-2.7.4.jar:?]
at com.sun.proxy.$Proxy46.retrieveMetadata(Unknown Source) [?:?]
at org.apache.hadoop.fs.CosFileSystem.getFileStatus(CosFileSystem.java:428) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem.processDeleteOnExit(FileSystem.java:1409) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem.close(FileSystem.java:2070) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.CosFileSystem.close(CosFileSystem.java:1160) [hadoop-cos-2.7.5-5.9.4.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2760) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer.run(FileSystem.java:2777) [hadoop-common-2.7.4.jar:?]
at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54) [hadoop-common-2.7.4.jar:?]

使用ssh代理后,https协议无法正常通讯

http正常使用,https报错如下图
image
代理配置如下
Established tunneling session to airbyte-oss-test-1258678410.cos.ap-nanjing.myqcloud.com:443. Port forwarding started on /127.0.0.1:42685
使用时配置了

            clientConfig.httpProxyIp = "127.0.0.1"
            clientConfig.httpProxyPort = port

调试过程中发现代理访问本地端口使用的是http,如图
image

手动注入为hppts后报错信息变更为

com.qcloud.cos.exception.CosClientException: Certificate for <127.0.0.1> doesn't match any of the subject alternative names: [*.cos.ap-nanjing.myqcloud.com, *.cn-east.myqcloud.com, *.cn-north.myqcloud.com, *.cn-south-2.myqcloud.com, *.cn-south.myqcloud.com, *.cn-southwest.myqcloud.com, *.cos-cdn.ap-nanjing.myqcloud.com, *.cos-cdn.ap-nanjing.tencentcos.cn, *.cos-cdn.ap-nanjing.tencentcos.com, *.cos-cdn.ap-nanjing.tencentcos.com.cn, *.cos-control.ap-nanjing.myqcloud.com, *.cos-control.ap-nanjing.tencentcos.cn, *.cos-control.ap-nanjing.tencentcos.com, *.cos-control.ap-nanjing.tencentcos.com.cn, *.cos-ext.ap-nanjing.myqcloud.com, *.cos-ext.ap-nanjing.tencentcos.com, *.cos-internal.accelerate.tencentcos.cn, *.cos-internal.accelerate.tencentcos.com.cn, *.cos-internal.ap-nanjing.myqcloud.com, *.cos-internal.ap-nanjing.tencentcos.cn, *.cos-internal.ap-nanjing.tencentcos.com, *.cos-internal.ap-nanjing.tencentcos.com.cn, *.cos.accelerate.myqcloud.com, *.cos.accelerate.tencentcos.cn, *.cos.accelerate.tencentcos.com, *.cos.accelerate.tencentcos.com.cn, *.cos.ap-nanjing.tencentcos.cn, *.cos.ap-nanjing.tencentcos.com, *.cos.ap-nanjing.tencentcos.com.cn, *.file.myqcloud.com, *.file.tencentcos.cn, *.file.tencentcos.com.cn, cos-cdn.ap-nanjing.myqcloud.com, cos-cdn.ap-nanjing.tencentcos.cn, cos-cdn.ap-nanjing.tencentcos.com, cos-cdn.ap-nanjing.tencentcos.com.cn, cos-control.ap-nanjing.myqcloud.com, cos-control.ap-nanjing.tencentcos.cn, cos-control.ap-nanjing.tencentcos.com, cos-control.ap-nanjing.tencentcos.com.cn, cos-ext.ap-nanjing.myqcloud.com, cos-ext.ap-nanjing.tencentcos.com, cos-internal.accelerate.tencentcos.cn, cos-internal.accelerate.tencentcos.com.cn, cos-internal.ap-nanjing.myqcloud.com, cos-internal.ap-nanjing.tencentcos.cn, cos-internal.ap-nanjing.tencentcos.com, cos-internal.ap-nanjing.tencentcos.com.cn, cos.accelerate.myqcloud.com, cos.accelerate.tencentcos.cn, cos.accelerate.tencentcos.com, cos.accelerate.tencentcos.com.cn, cos.ap-nanjing.tencentcos.cn, cos.ap-nanjing.tencentcos.com, cos.ap-nanjing.tencentcos.com.cn, file.myqcloud.com, file.tencentcos.cn, file.tencentcos.com.cn, cos.ap-nanjing.myqcloud.com]
	at com.qcloud.cos.utils.ExceptionUtils.createClientException(ExceptionUtils.java:47) ~[cos_api-5.6.153.jar:?]
	at com.qcloud.cos.http.DefaultCosHttpClient.executeRequest(DefaultCosHttpClient.java:589) ~[cos_api-5.6.153.jar:?]
	at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:483) ~[cos_api-5.6.153.jar:?]
	at com.qcloud.cos.COSClient.invoke(COSClient.java:614) ~[cos_api-5.6.153.jar:?]
	at com.qcloud.cos.COSClient.listObjects(COSClient.java:1961) ~[cos_api-5.6.153.jar:?]

关于 copyobject 的疑问

拷贝方法执行完未报错,就说明文件已经拷贝过去了吗?
此时我马上去下载新产生的文件,可以保证文件一定正确不?
我copy完以后去调取文件元数据的getContentMD5 发现是空的

求助>>DefaultCosHttpClient:369 抛空指针异常

version: 5.5.5

org.apache.http.client.ClientProtocolException: null
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:369)
	at com.qcloud.cos.COSClient.invoke(COSClient.java:530)
	at com.qcloud.cos.COSClient.putObject(COSClient.java:848)
	at com.innjoy.pms.middleware.util.UploadFileUtil.upload(UploadFileUtil.java:61)
	at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.uploadImg(IdentificationMigrationServiceImpl.java:148)
	at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.getImgUrl(IdentificationMigrationServiceImpl.java:107)
	at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl.saveIdentification(IdentificationMigrationServiceImpl.java:67)
	at com.innjoy.pms.middleware.service.impl.IdentificationMigrationServiceImpl$$FastClassBySpringCGLIB$$810649c.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:108)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	... 22 common frames omitted
Caused by: java.net.SocketException: Connection reset by peer: socket write error
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
	at org.apache.http.impl.io.SessionOutputBufferImpl.streamWrite(SessionOutputBufferImpl.java:124)
	at org.apache.http.impl.io.SessionOutputBufferImpl.flushBuffer(SessionOutputBufferImpl.java:136)
	at org.apache.http.impl.io.SessionOutputBufferImpl.write(SessionOutputBufferImpl.java:167)
	at org.apache.http.impl.io.ChunkedOutputStream.flushCacheWithAppend(ChunkedOutputStream.java:122)
	at org.apache.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:179)
	at org.apache.http.entity.InputStreamEntity.writeTo(InputStreamEntity.java:134)
	at org.apache.http.impl.execchain.RequestEntityProxy.writeTo(RequestEntityProxy.java:121)
	at org.apache.http.impl.DefaultBHttpClientConnection.sendRequestEntity(DefaultBHttpClientConnection.java:156)
	at org.apache.http.impl.conn.CPoolProxy.sendRequestEntity(CPoolProxy.java:160)
	at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:238)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	... 24 common frames omitted

关于cosClient shutdown方法

是每次调用完一个方法都要执行一次shutdown吗?
我目前是这样:

    COSClient cosclient = new COSClient(cred, clientConfig);
    try {
      cosclient.upload(...);
      cosclient.upload(...);
      cosclient.upload(...);
      ...
    } finally {
      cosclient.shutdown();
    }

是否合理。

生成在 URL 可使用的授权参数

使用场景:
后端生成 authorizationStr,前端利用这个 authorizationStr 操作 cos 资源。

目前sdk提供:
COSSigner.buildAuthorizationStr() 可以用来生成 authorizationStr,但是直接用到 URL 的 sign 参数中会出现Access denies。只能放在 HTTP 请求的 Authorization header才能有权访问资源。

要想使用在 URL 加 sign 参数的方式来用这个 authorizationStr,必须要做一下以下操作:
UrlEncoderUtils.encode(authorizationStr)

那么,能不能提供一个现成的 API 来生成这个可以用在 URL 上的授权参数呢?
比如在 COSSigner 中加上以下方法:

public String sign(HttpMethodName methodName, String resouce_path,
            COSCredentials cred, Date expiredTime) {
    String authorizationStr = buildAuthorizationStr(methodName, resource_path, cred, expiredTime);
    return UrlEncoderUtils.encode(authorizationStr);
}

COS文件上传后,再调用getObjectMetadata,得到的Etag有误

文件是通过 spark 直接导入到 cosn://buc-13424123/key1/da/pp1/part-00000-f124f8f4-5b44-4c83-aece-6cd20629860b-c000.snappy.parquet
获取到的 ETag 为:001a387d4ca31d1952d3409c63503c71-3
得到的 ETag 位数超过32位,后面的 “-3” 去掉也不是正确的校验值。

分片上传时,第二个分片上传超时.

使用sdk进行分片上传时,第一个分片能上传,
第二个分片上传时就报超时错误。

普通上传没有问题。

Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:426)
	at com.qcloud.cos.http.DefaultCosHttpClient.executeRequest(DefaultCosHttpClient.java:586)

COS SDK 有依赖冲突

执行命令

mvn dependency:tree -Dverbose

出现了okio:jar:1.6.0:compile - omitted for conflict with 1.12.0

[INFO] ------------------------------------------------------------------------
[INFO] Building cos-java-sdk 5.6.42
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ cos_api ---
[INFO] com.qcloud:cos_api:jar:5.6.42
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.3:compile
[INFO] |  +- org.apache.httpcomponents:httpcore:jar:4.4.6:compile
[INFO] |  \- commons-logging:commons-logging:jar:1.2:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.7.26:compile
[INFO] +- commons-codec:commons-codec:jar:1.10:compile
[INFO] +- joda-time:joda-time:jar:2.9.9:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.1:compile
[INFO] |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.1:compile
[INFO] |  \- com.fasterxml.jackson.core:jackson-core:jar:2.12.1:compile
[INFO] +- com.tencentcloudapi:tencentcloud-sdk-java-kms:jar:3.1.213:compile
[INFO] |  \- com.tencentcloudapi:tencentcloud-sdk-java-common:jar:3.1.213:compile
[INFO] |     +- (commons-logging:commons-logging:jar:1.2:compile - omitted for duplicate)
[INFO] |     +- com.squareup.okhttp:logging-interceptor:jar:2.7.5:compile
[INFO] |     |  \- com.squareup.okhttp:okhttp:jar:2.7.5:compile
[INFO] |     |     \- (com.squareup.okio:okio:jar:1.6.0:compile - omitted for conflict with 1.12.0)
[INFO] |     +- com.squareup.okio:okio:jar:1.12.0:compile
[INFO] |     +- com.google.code.gson:gson:jar:2.2.4:compile
[INFO] |     \- javax.xml.bind:jaxb-api:jar:2.3.0:compile
[INFO] +- junit:junit:jar:4.13.1:test
[INFO] |  \- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO] +- org.bouncycastle:bcprov-jdk15on:jar:1.64:compile
[INFO] +- com.qcloud:qcloud-java-sdk:jar:2.0.6:test
[INFO] \- com.tencent.cloud:cos-sts-java:jar:3.0.8:test
[INFO]    \- org.json:json:jar:20180130:test

关于使用“COS 持久化处理”接口遇到的问题

新的文档中持久化处理一节中,说:

您只需要在请求包头部中加入 Pic-Operations 项并设置好相应参数,就可在图片上传时实现相应的图片处理,并可将原图和处理结果存入到 COS。
图片上传的请求包与 COS V5 简单上传文件接口一致,只在请求包头部增加图片处理参数。

也提到了,你处理的后的结果是通过 http Response 的 content 中的 UploadResult 节点返回:

<UploadResult>
  <OriginalInfo>
  ...
  </OriginalInfo>
  <ProcessResults>
  ...
  </ProcessResults>
</UploadResult>

如果直接使用 CosClient 的 putObject 上传文件,经过 CosMetadataResponseHandler 处理,Response 的 content 就丢弃了,也就是说,无法通过 putObject 获取 Pic-Operations 的处理结果,只能自己全部实现整个上传过程。这个问题能修复下么?

Connection pool shut down

25升级到28后,运行一天左右.就会大面积报下面的错误

com.qcloud.cos.exception.CosClientException: httpClient execute occur a unknow exception, httpRequest: endpoint: xxx-123456.cos.ap-shanghai.myqcloud.com, resourcepath: /projects/A__2020-03/project_5e67509360b2bcd915725ae9/task_5f27742a60b223ab27ac5b42/, httpMethod: HEAD, headers { Authorization : q-sign-algorithm=sha1&q-ak=sssss&q-sign-time=1596589498;1596593098&q-key-time=1596589498;1596593098&q-header-list=host&q-url-param-list=&q-signature=4d66cb9db906caa2f4c4711bb624d0afb6dc7513, User-Agent : cos-java-sdk-v5.6.28/Linux/jdk-1.8.0_151/Java HotSpot(TM) 64-Bit Server VM, Host : xxx-123456.cos.ap-shanghai.myqcloud.com, }, params: { }

	at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:501)
	at com.qcloud.cos.COSClient.invoke(COSClient.java:601)
	at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1211)
	at com.qcloud.cos.COSClient.getObjectMetadata(COSClient.java:1188)
	at com.qcloud.cos.COSClient.doesObjectExist(COSClient.java:1175)
	at java.lang.Thread.run(Thread.java:748)

Caused by: java.lang.IllegalStateException: Connection pool shut down
	at org.apache.http.util.Asserts.check(Asserts.java:34)
	at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:251)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:175)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at com.qcloud.cos.http.DefaultCosHttpClient.executeOneRequest(DefaultCosHttpClient.java:414)
	at com.qcloud.cos.http.DefaultCosHttpClient.exeute(DefaultCosHttpClient.java:475)
	... 22 more

Key 包含空格导致GenerateDocPreviewUrl 出现SignatureDoesNotMatch

SDK语言: Java SDK
SDK版本号: 5.6.167
Key 包含空格导致GenerateDocPreviewUrl 出现SignatureDoesNotMatch

image

image

DocHtmlRequest 中的key 如果包含空格这句代码会报错:URIBuilder uriBuilder = new URIBuilder(urlStr)。
将空格替换为 “%20”,又会导致SignatureDoesNotMatch:因为 buildAuthorizationStr 函数接收的key的空格被替换为“%20”了。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.