depscloud / gitfs Goto Github PK
View Code? Open in Web Editor NEWVirtual file system for managing your git integrations
License: MIT License
Virtual file system for managing your git integrations
License: MIT License
I noticed that after making a change, there was a lock lingering around. Seems like something is off.
[mjpitz@mjpitz ~/Development/code/github.com/mjpitz/mjpitz.com master 1/1]
$ git add .
fatal: Unable to create '/Users/mjpitz/Development/code/github.com/mjpitz/mjpitz.com/.git/index.lock': File exists.
Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.
Currently pointed at mjpitz
Right now we only support parsing ssh urls. We should add support to parse https urls.
This may be two or one issues - split it if needed.
$ cat ~/.gitfs/config.yml:
mount: "${HOME}/GitFS"
accounts:
- github:
oauth2:
token: ...
users:
- peter-kehl
clone:
repositoryRoot: "${HOME}/.gitfs/cache" # this is where your repos will be cloned
depth: 0 # depth 0 means full clone
$ ~/Downloads/gitfs_linux_amd64 start
INFO[0000] [main] configured mount point: /home/pkehl/GitFS
INFO[0000] [main] fetching repositories
...
NFO[0005] [main] now serving file system
ERRO[0026] [filesystem.directory] failed to clone url: [email protected]:peter-kehl/docker.github.io.git, failed to clone repo: [email protected]:peter-kehl/docker.github.io.git: ssh: handshake failed: knownhosts: key is unknown
That did create a directory structure under ~/GitFS, with my repositories - both my own and in my organisations. However, all are empty:
$ ls /home/pkehl/GitFS/github.com/peter-kehl/4clojure
# -> empty output
$ grep 4clojure ~/.gitfs/cache/*/.git/config
N19UKEAUM8PTKEEIH4CRI8Q5HE9FIHVPUN99BR49MQPJ7DE6ME6G/.git/config: url = [email protected]:peter-kehl/4clojure.git
$ ls -a ~/.gitfs/cache/N19UKEAUM8PTKEEIH4CRI8Q5HE9FIHVPUN99BR49MQPJ7DE6ME6G
. .. .git
$ ~/Downloads/gitfs_linux_amd64 stop
INFO[0000] Unmounting /home/pkehl/GitFS
That shows the same even when repeated.
However, it didn't kill the server process. Even worse, after I Ctrl+C stopped the main process, I couldn't restart it:
$ ~/Downloads/gitfs_linux_amd64 start
INFO[0000] [main] configured mount point: /home/pkehl/GitFS
INFO[0000] [main] fetching repositories
INFO[0000] [remotes.github] processing organizations for user: peter-kehl
INFO[0000] [remotes.github] processing repositories for user: peter-kehl
INFO[0001] [remotes.github] processing repositories for organization: Melati
INFO[0001] [remotes.github] processing repositories for organization: SeLite
INFO[0002] [remotes.github] processing repositories for organization: VanClojure
INFO[0003] [remotes.github] processing repositories for organization: DaisyDiff
INFO[0003] [remotes.github] processing repositories for organization: PanEris
INFO[0003] [main] parsing repositories into a directory structure
INFO[0003] [main] attempting to mount /home/pkehl/GitFS
2019/01/28 09:11:40 mount helper error: fusermount: failed to access mountpoint /home/pkehl/GitFS: Permission denied
ERRO[0003] [main] failed to mount mountpoint: fusermount: exit status 1
Why is that worth fixing? A user is likely to start gitfs right after rebooting. On laptops, some Linux distributions (Manjaro/Arch) start with disabled Wifi. That means the user user has to reboot again...
Right now, we don't tell people how to contribute to the project and we should.
[jpitz@jpitz ~/Development/code/github.com/mjpitz/mjpitz.com master 0/0]
$ git add _config.yml
fatal: error when closing sha1 file: Input/output error
Nothing coming through on the error logs so definitely need to add some better logging to help track this down.
Info | |
---|---|
os | darwin |
fs | osfs |
Implement the start
command as described in #12
Right now, we don't use a linter and we really should. I know there is a good number of things that will fail the linters check. Todo:
I believe that this is because we don't have a proper inode system in place.
When I run something like jekyll serve
, files aren't automatically updating. This is likely because the modification times aren't being updated accordingly.
You can find a copy of many github repositories on bigquery using the open github dataset. Let's add a remote that can support querying bigquery for these repositories.
Right now, we require third party dependencies to be installed on the developer machine. Let's encapsulate those into a docker build image so that engineers can develop without needing to install dev-specific tooling.
During gh-19, I had refactored the cloning logic and left room for the addition of svn and hg repo management. In this issue, let's implement the hgcloner
. I'd start by looking at the existing gitcloner
and compositecloner
implementations.
This has fallen a fair bit behind compared to some of the older projects. We will need to go through and update this project to use some of the latest libraries. There will be a fair number of configuration changes that need to be made.
Right now, we don't communicate how to gracefully stop the server. We should add a section to the readme.
gox won't be able to compile it, but we can do it using go build
directly.
There's not much there right now: https://mjpitz.github.io/gitfs/
It's currently being generated off of the /docs directory.
I started a docs directory that is currently used to power https://mjpitz.github.io/gitfs. In this ticket, let's get a project site up and running.
Should probably do #20 before completing this.
Set up a timed interval to re-pull remotes and add new projects where necessary. We should never delete projects form the local system as they may contain changes and we want to give engineers the opportunity to update their references where necessary.
Dear Mya,
Thank you for GitFS.
Please, could you document scope(s) of GitHub token. It seems that "read:user" scope is enough.
The configuration should already be defined and there is a placeholder in the pkg/remotes/remotes.go
file for adding the construction of the bitbucket client. I was basing the config off of go-bitbucket.
This should let us use travis to produce binaries for the proper platforms. We can use gox to do cross compilation and we can target the appropriate platforms.
Right now, when we discover and parse a url, each node in the directory gets assigned a new inode id with each iteration. While this isn't a problem for our current usage, it will pose a bigger issue if we ever want to update the directory tree on the fly.
The configuration should already be defined and there is a placeholder in the pkg/remotes/remotes.go file for adding the construction of the gitlab client. I was basing the config off of go-gitlab.
Many projects have a single install.sh script that can be curl'd and piped into bash to install their application. We should add one to make it easy for folks to install without needing to install go.
Also update documentation accordingly.
Right now, we rely on your ssh keys to be added to the session. When gitfs starts up, the terminal won't have the ssh keys to work with and and so all clones will fail.
Right now, we always checkout at HEAD. It would be nice to be able to maintain a notion of a "stable" position for each repo.
We will need a pretty large design effort to get this kind of feature built in.
Right now, we only support leveraging the memfs implementation for the go-git integration. In this change we should handle a couple things.
This may be an issue with the underlying library, but I made a whole change on a repo and then went to push it and ran into this issue. At first I thought it was the integration with gitfs, but when I looked at the underlying directory, it has the same issue.
[mjpitz@mjpitz ~/.gitfs/cache/DA0KGNSRU2DH2QGP0N4INC1225L3ALUVUJNED5PK7JTJTJ8I00B0 master 1/1]
$ git push
fatal: No configured push destination.
Either specify the URL from the command-line or configure a remote repository using
git remote add <name> <url>
and then push using the remote name
git push <name>
When I'm on a "user" directory, an ls
causes each repo to be cloned.
Was trying to go through a few projects gh-pages. After doing a sed operation, the permissions of a file appear to be stripped:
$ ls -la _layouts/
total 32
drwxr-xr-x 4 jpitz staff 128 May 2 06:46 .
drwxr-xr-x 13 jpitz staff 416 May 2 06:45 ..
---------- 1 jpitz staff 5177 May 2 06:46 default.html
-rw-r--r-- 1 jpitz staff 4228 May 2 06:48 reference.html
Not sure what happened there but should be easy to reproduce:
$ ag -l <pattern> | xargs sed -i "s:<pattern>:<replace>:g"
Right now, we clone on the fly. Some people would like to have some repos cloned automatically. Seems like adding support for this would be easy. Under the clone config, we can add an init section that has a list of strings / regex. If the repo url matches the string or regex, then the repo is added to a list to be cloned once the fs starts up.
During gh-19, I had refactored the cloning logic and left room for the addition of svn and hg repo management. In this issue, let's implement the svncloner
. I'd start by looking at the existing gitcloner
and compositecloner
implementations.
Right now we only support shallow clones. Lets work through the requirements for adding support for non-shallow clones and the configuration that we would like to override on a per repository basis. I.e some repos you may never want to full clone. Default should be to shallow clone to keep the fs responsive.
Right now, there is a single command gitfs
. This starts a file system server on a designated mountpoint. Ideally, we want to support a variety of commands, not just starting the server. For example:
start
would perform the same functionality as gitfs
does today. If no command is provided, then this command should be invoked by default.stop
would force the file system server to unmount. Note, that using fuse.Unmount
will not work as it requires all references to the underlying files to be released (i.e all terminal windows and applications closed). I've been able to do this on osx and linux using the sudo umount -f ${mountpoint}
config
would behave a lot like git-config, making it possible to modify properties without needing to modify the files directly.Each of these commands should be implemented as their own ticket.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.