dingmib / fqueue Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/fqueue
License: Apache License 2.0
Automatically exported from code.google.com/p/fqueue
License: Apache License 2.0
--------------------------------------------------
FQueue fQueue = new FQueue("E:/q/db/");
StringBuilder sb = new StringBuilder();
int length = Integer.parseInt(args[0]);
for (int i = 0; i < length; i++) {
sb.append("a");
}
byte[] data = sb.toString().getBytes();
for (int i = 0; i < 1000000; i++) {
fQueue.add(data);
}
for (int i = 0; i < 1000000; i++) {
byte[] b = fQueue.poll();
if(b!=null)
System.out.println(i + "-");
}
--------------------------------------------------
问题: add完,有4个db文件,但是poll完,还剩下fqueuedata_2.idb
和fqueuedata_4.idb这两个文件。poll完,文件是否应该都清除呢?
Original issue reported on code.google.com by [email protected]
on 1 Dec 2011 at 10:03
您好,最近准备使用Fqueue,但是发现个问题,长度一旦太长se
t进去的和get的不一样,请问这个是怎么回事
Original issue reported on code.google.com by [email protected]
on 20 Nov 2012 at 9:41
when i add message to fqueue this exception was thrown.
java.lang.NullPointerException
at com.google.code.fqueue.log.LogEntity.write(LogEntity.java:205)
at com.google.code.fqueue.FSQueue.add(FSQueue.java:150)
at com.google.code.fqueue.FQueue.offer(FQueue.java:66)
at com.google.code.fqueue.FQueue.offer(FQueue.java:1)
at java.util.AbstractQueue.add(AbstractQueue.java:68)
and
java.lang.IllegalStateException: Queue full
at java.util.AbstractQueue.add(AbstractQueue.java:71)
at com.tenddata.collector.util.FSQueueWriter.run(FSQueueWriter.java:39)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
i had none configuration for the queue, and disk was not full.
Original issue reported on code.google.com by [email protected]
on 10 Dec 2013 at 4:10
1. 在程序里面New一个FQueue对象
2.
在空闲一段时间队列未使用的时候调用FQueue对象的close方法关
闭队列
3. 关闭后有两个线程任然未退出,并且不是Daemon模式的线程
应该在调用FQueue对象的close方法后自动中断线程
此问题在JDK1.6以上版本(以下版本未测试)、redhat linux
、windows下均存在
经源码跟踪发现时FSQueue里面的有一个deleteFileRunner在以一种死
循环的方式运行,导致Executors.newSingleThreadExecutor()无法退出,��
�一个线程是LogEntity里面的的一个Executor,是一样的现象.
Original issue reported on code.google.com by [email protected]
on 27 Jan 2014 at 2:26
有没有发现 用FSQueue2个线程,一读一写 会报异常
Exception in thread "Thread-1" java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:501)
at java.nio.DirectByteBuffer.putInt(DirectByteBuffer.java:647)
at com.google.code.fqueue.log.LogIndex.putWriterPosition(LogIndex.java:113)
at com.google.code.fqueue.log.LogEntity.putWriterPosition(LogEntity.java:176)
at com.google.code.fqueue.log.LogEntity.write(LogEntity.java:214)
at com.google.code.fqueue.FSQueue.add(FSQueue.java:150)
at com.google.code.fqueue.FSQueue.add(FSQueue.java:139)
Original issue reported on code.google.com by [email protected]
on 12 Apr 2012 at 3:16
What steps will reproduce the problem?
1. 通过xmemcache客户端放java对象
2. 再通过xmemcache客户端取出
3. 转类,提示不能把字符串转为x类型
最后发现xmemcache客户端在向fqueue
server发送消息前已正常序列化,但fqueue
get返回后xmemcache无法识别类型,从而没能反序列直接做string��
�理....
根据http://www.blogjava.net/kilglme2008/archive/2011/05/31/351447.html和看��
�码应该是flags的问题,fqueue也没有维持flags所以靠flags来序列�
��的客户端都不能直接保存类似对象的类型了,最后尝试了下
,应该可以保存的但不知是否对性能影响过大 - - patch见附件
ps:其实自己转json后存也很方便,但如果走memcache不推荐直接
存object还是在文档中说明下哈,这样大家可以少走弯路,thx!
Original issue reported on code.google.com by [email protected]
on 23 Nov 2011 at 5:53
Attachments:
单独使用Fqueue时,设置文件太小,如:
FQueue fQueue = new FQueue("xxxx",100*1024);
写到2,3个文件后就会抛异常
翻看源码后看每次新生成idb文件是有一个单独线程在运行,��
�步怀疑是上一个文件写满,下一个文件还未生成造成的,不�
��道怀疑的对不对
异常信息如下:
com.google.code.fqueue.exception.FileFormatException: file format error
at com.google.code.fqueue.log.LogEntity.<init>(LogEntity.java:89) --这句也可能在79行抛出,都出现过
at com.google.code.fqueue.FSQueue.createLogEntity(FSQueue.java:110)
at com.google.code.fqueue.FSQueue.rotateNextLogWriter(FSQueue.java:126)
at com.google.code.fqueue.FSQueue.add(FSQueue.java:152)
at com.google.code.fqueue.FQueue.offer(FQueue.java:66)
at com.google.code.fqueue.FQueue.offer(FQueue.java:1)
at java.util.AbstractQueue.add(AbstractQueue.java:68)
at TestFQueue2.main(TestFQueue2.java:27)
java.lang.IllegalStateException: Queue full
at java.util.AbstractQueue.add(AbstractQueue.java:71)
at TestFQueue2.main(TestFQueue2.java:27)
Original issue reported on code.google.com by [email protected]
on 22 Mar 2013 at 6:55
索引文件没有定时通过force()方法强行写入硬盘, LogIndex类的close()方法也没有被调用, 进程被kill或断电应该会造成索引文件记录读写位置和数据文件的记录不一致.
请问sunli关于索引文件和数据文件一致性的问题你是如果考虑的.
Original issue reported on code.google.com by [email protected]
on 8 Jun 2012 at 9:40
--------------------------------------------------------------------
public FSQueue(String dir, int fileLimitLength) throws Exception {
this.fileLimitLength = fileLimitLength;
File fileDir = new File(dir);
if (fileDir.exists() == false && fileDir.isDirectory() == false) {
if (fileDir.mkdirs() == false) {
throw new IOException("create dir error");
}
}
---------------------------------------------------------------------
问题:如果dir是一个已经存在的文件路径,如“E:/a/b.txt”;�
��么就会有问题!抛出
“java.io.IOException: 系统找不到指定的路径。”
Original issue reported on code.google.com by [email protected]
on 1 Dec 2011 at 5:10
文档能否比较详细说明是如何设计的,
跟类似的项目是否有对比实验.
Original issue reported on code.google.com by [email protected]
on 27 Aug 2011 at 2:01
直接拿项目介绍里的代码跑了一下 WINDOWS系统 设了1G内存
写1024字节
发现LogEntity里调用mappedByteBuffer.force();操作,有时候巨卡,多�
��时候达到1分多钟 求解
public static void main(String[] args) throws Exception {
FQueue fQueue = new FQueue("/home/q/db/");
StringBuilder sb = new StringBuilder();
int length = Integer.parseInt(args[0]);
for (int i = 0; i < length; i++) {
sb.append("a");
}
byte[] data = sb.toString().getBytes();
fQueue.add(data);// 预热一下
long start = System.currentTimeMillis();
for (int i = 0; i < 100000000; i++) {
fQueue.add(data);
}
System.out.println(100000000.0 / ((System.currentTimeMillis() - start) / 1000) + "qps");
fQueue.close();
}
Original issue reported on code.google.com by [email protected]
on 20 Jan 2015 at 8:56
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.