Comments (24)
from kbfs.
@mortenkjarulff did you mean to file this in keybase/client? Chat is pretty much completely separate from kbfs at this point.
from kbfs.
Hi,
Appears not to be so separate 😀
Shall I file it there instead?
/Morten
from kbfs.
Oh, I guess I glossed over or didn't understand the line about "outcomment the ls".
Ok, here's what appears to be the issue. The team folder you're ls
ing has been written to by a device that has since been revoked. When you ls
it, that triggers kbfs to attempt to "garbage-collect" the old data in it, by iterating through the old revisions of the folder. When it looks at the revisions that were written by the now-revoked device, it triggers a bunch of extra checking to verify that the device was valid at the time of the revision.
It looks like, in the case of this folder, the revisions by that revoked device are so expensive that they are slowing down chat.
Can you just ls
that folder once, and maybe let the garbage collection run overnight and see if it finishes? You can tell if it made progress overnight by looking at the LastGCRevision
field in the team folder's .kbfs_status
file.
If it doesn't make any progress, even after many hours of work, I'll try to figure out a custom solution for you.
And obviously we need to figure out a way to reduce these extra checks without affecting security...
from kbfs.
Not sure I understand. I can reproduce at any time, even long time after I revoked.
I will manually ls now, and start the script tomorrow.
from kbfs.
We only garbage-collect data when it is at least two weeks old, but it could be longer depending on when your devices were online and when they accessed this particular folder. In your case the device was revoked on October 3rd, but it's trying to garbage-collect data written on September 29th.
from kbfs.
Ok, will try
Log after manual ls
a0cb1d208188c2886665501c
from kbfs.
LastGCRevision": 0,
Is that good?
from kbfs.
That means it has never been successfully garbage collected. If the team/folder is very old, this is surprising and might indicate some other bug.
from kbfs.
Timestamp on folder is October 8, that could well be the date I created the subteam
from kbfs.
Looks like it was created on September 20, so that lines up with the garbage collection and revocation messages I mentioned above.
from kbfs.
LastGCRevision": 0,
d46b49510feabaac7be4161c
If it can help
from kbfs.
Please just let it run for a long time uninterrupted.
from kbfs.
Still 0
from kbfs.
Hrm that's a shame. I'm going to work today on making this whole thing better.
from kbfs.
I put up #1893 to help eliminate much of the redundant work we're doing for revoked devices. It's under review now, but if you're feeling bold and want to try it out before it's merged officially, let me know.
from kbfs.
When will it be official maybe?
If it can help, how can I test it?
Would it help me if I created a new subteam, copy over and rename?
/Morten
from kbfs.
It probably won't be out in the official release for a while, but I've made a .deb file for you, if you feel like trying it out:
/keybase/public/strib/keybase-2.10.0-20181102162247.e4a1860091-amd64.deb
or if you trust keybase.pub:
https://keybase.pub/strib/keybase-2.10.0-20181102162247.e4a1860091-amd64.deb
Note this is not a well-tested release, so there could be some rough edges. Also, there might be multiple issues going on with your folder, and I only fixed the most obvious first one, so this debugging might take more steps. We'll see!
If you feel like trying it, please install the above deb, run run_keybase
, ls
your folder again, and wait a bit. If you do a log send after a few minutes I can check to see if it seems to be making progress. Thanks!
from kbfs.
Installed your test version.
Still 0.
921efd3e424dcbad89bf081c
/Morten
from kbfs.
Thanks. I was wrong to say that it starts the GC process after 1 minute -- we changed it a while back to one hour. It looks like you sent the log less than an hour after you did the ls
. Can you please send another log now if it's been up this whole time?
from kbfs.
"LastGCRevision": 9,
d96ca4acddd6f20834cd6d1c
It has been running for some hours.
/Morten
from kbfs.
Ok cool. It at least stopped chewing up your CPU right? I'll look into why it hasn't gone past 9 yet tomorrow, but that seems like a separate issue.
from kbfs.
Have been running my test script all day and send/read times are constant low.
I don't think the issue was CPU, but can be wrong.
Should I reinstall the official version again, or?
/ Morten
from kbfs.
Great!
I wouldn't recommend re-installing the master version, as the problem could come back. Just wait for the next apt release, which will include these changes. Though if you seem to be having unexpected problems with the release you have. let me know.
from kbfs.
Related Issues (20)
- Ability to Nuke Database HOT 1
- KBFS file size limit of 512 KiB breaks git-remote-keybase HOT 9
- /keybase "resets" at random times HOT 15
- kbfs team folder "split" HOT 4
- fusermount: entry for /run/user/1000/keybase/kbfs not found in /etc/mtab HOT 13
- CreateFile not call when file name contains colon? HOT 5
- Removing "files before account reset" folder fails HOT 8
- android: git-remote-keybase stderr dup error HOT 12
- Keybase folder is empty HOT 2
- Question: How to use kbfs directly with snap applications? (Ubuntu 18.04.1 LTS) HOT 4
- Truncate file on Linux dosn't always work HOT 6
- Relative paths error on MacOS HOT 2
- Latest update of KBFS eating up a bunch of memory on macOS again. HOT 10
- dokan: path does not exist HOT 7
- Is there an obvious reason why I'm having trouble monitoring kbfs with incrond? HOT 8
- Oops. HOT 1
- Enormous KBFS memory usage when not doing anything (Arch Linux) HOT 10
- Extremely slow startup with VPN HOT 5
- incorrect quota state HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kbfs.