TFS Branching about git-tfs

[spraints]

It's not supported right now. I never had any TFS branches that I wanted to convert to git branches. I'll add it to the wishlist.

Do you use a TTB layout, like you would with SVN?

[CoreyKaylor]

We've started to use git-tfs recently and this is one area where I feel like the workflow would be much improved with something like this. For us we use a layout similar to TTB.

[LievaartR]

After some googling for the term TTB, I think we use a similar structure, a main branch(trunk) a dev directory for development branches, and a release subdirectory that contains branches for versions we have released, or are about to release.

The project is still developed quite heavily, but there are also released versions, and I find it hard in TFS to see which commit is for which branche, and what changes were pulled in by a merge.

[rbanks54]

Because branches are so different between the two systems (TFS uses folder based and git is revision based) there's some challenges in doing this.

If I make a clone of a TFS repository from the root (i.e. $/ProjectName), would you expect the content from all the branches to come down into the default git branch (as per current behaviour), or for branched folders to be ignored?

Would you expect to be able to push a local git branch to TFS? I'm hoping not.
Would you expect git merges across branches to be reflected back in TFS? Which implies that baseless merges across branches are required.
TFS2008 supports nested branches, TFS2010 doesn't (new branching model). Do you expect branching to work across both versions of TFS? And in the same way?

P.S. I'm just asking questions to try and clarify what people's expectations would be. :-)

[CoreyKaylor]

My answers inline.

If I make a clone of a TFS repository from the root (i.e. $/ProjectName), would you expect the content from all the branches to come down into the default git branch (as per current behaviour), or for branched folders to be ignored?

I would say ignored. Or I would think that in order for this to work right I would clone from $/ProjectName/trunk which would then be able to with the clone pull in the branches as branches in git. Ignoring would happen automatically because the branch would be located in something like $/ProjectName/branches/feature1

Would you expect to be able to push a local git branch to TFS? I'm hoping not.

No, I wouldn't expect this.

Would you expect git merges across branches to be reflected back in TFS? Which implies that baseless merges across branches are required.

That might be nice, but I wouldn't think of it as a necessity. Might also lose some of the history / auditability tfs gives you for a merge were this to be built. If so, I couldn't encourage its use.

TFS2008 supports nested branches, TFS2010 doesn't (new branching model). Do you expect branching to work across both versions of TFS? And in the same way?

We only use 2010, so I don't care about 2008 :P

The biggest benefit IMO is just being able to issue a "git checkout branchname" and that branch name would have the ability to "git tfs ct" and the check-in would be against the correct branch.

[spraints]

If you make a clone of a TFS repository "$/Project", I would expect everything to be included. I have, in the past, created files in TFS by branching from a similar file. It may not have been the best idea, but it would be annoying to do that and not have git-tfs available. Also, it would require quick-clone to do more work to find branches that it should leave out.

If you make a clone of a TFS dir "$/Project/trunk" and tell git-tfs that branches are in "$/Project/branches", then it would be nice if git-tfs were to set up the corresponding branches. I'm not sure if it would be able to get information about TFS merges, or if it did if it would be meaningful in the context of git. (This is similar to the git-svn init -T/-b options.)

I don't know enough about the 2010 branching model changes to be able to say how it would fit.

[CoreyKaylor]

What you've described would be perfect. An opt in branches directory would be perfect. I don't think the TFS merges make a lot of sense to pull into git, other than recognizing that there were changes and pulling down new files which would come as normal anyway.

[jimitndiaye]

Is there any work being planned to address this issue or has it already been implemented?

[mika76]

Any news of this? personally I would love to get any branches down as git branches...

[dennismi]

We are not using a TTB.

We have 1 branch which (in theory) should always be releasable (called dev\source). From there we made a branch to release (only the one - should always only contain the current released code) and then we have a branch for each developer (5 atm) but they all stem from dev\source. So in my scenario i would like it to create the branches as it sees them and keep the updating them when merges happen.

To make matters a bit worse, we actually merge both to and from our dev\source, both fixes made in our release branch and new features coming in from our development branches.

I might even settle for allowing to do some git-tfs magic like (a more manually approach)

git tfs branch-clone http://path_to_tfs $/path/to/branch/target/folder <branchname>

then it should probably create the git branch and the correct sha1 and start pushing the changeset involved into this new branch that would involve figuring out what tfs changeset the branch occurred... this is only problematic if you want to have the right merge tracks in git (which you kinda would : - )

a really nice feature would be to be able to check in changes in the git-branches to the corresponding TFS branches...

some fuel for thought - that might even have been considered...

[sc68cal]

One corner case we need to be aware of that is present in some repository layouts (read: Me) : Where a developer only has permissions on a branched directory - but not the origin.

[mika76]

We use the Scenario # 2: Concurrent Hotfix, Service Pack, and vNext from http://vsarbranchingguide.codeplex.com/

You're right about the security issues, but maybe that could be sorted by the authors file mentioned in issue #118 ? Also it would depend on the reasons the clone using git. One of the reasons I would like to use it is for simple backup. TFS Backup and restore is a laborious process, so a git backup which would be up and running in no time would help a lot. Assuming this, then the clone would be done by an admin and security would not be an issue.

Personally I would love to see clones of branches not as separate directories but as proper git branches. Security should be handled, but if you don't have access, could a branch be ignored? Would that create some sort of inconsistent clone?

Also, security can be done on folder level and not just branch level. How is this handled currently?

[sc68cal]

You're right about the security issues, but maybe that could be sorted by the authors file ?

I see where you're going with that - but really the authors file just maps different names to display, it wouldn't handle
any sort of authentication.

Personally I would love to see clones of branches not as separate directories but as proper git branches. Security should be handled, but if you don't have access, could a branch be ignored? Would that create some sort of inconsistent clone?

It's more likely that we wouldn't be able to get any metadata about which changeset the branch began from, if the API calls return similar results as the screenshot I gave. So we wouldn't have enough context to build up a git branch, so we'd have to skip it I guess.

Also, security can be done on folder level and not just branch level. How is this handled currently?

I don't think we have any code to handle security - I think we just take whatever changesets the server will give us. I may be wrong though.

[dennismi]

I haven't checked but what does a given changeset tell about branching, that could be a way to determin when to branch, that is

get changeset
check metadata about branching
if branch found create git branch
checkout git branch - and commit changeset

but now it gets tricky when are we back on the master branch again, and what about changesets that span several TFS branches (it could occur - i know we got some changesets that is "global")

[sc68cal]

In my Notes I have a link to a example on SO where you can get information about a branch.

[dennismi]

kewl, so the information is there, but there are SOO many scenarios to consider :/ ...

I wonder if i should clone the repo and start hacking, unless someone is really close to completing it :)

[sc68cal]

Join us on IRC - #git-tfs on freenode - I'd be happy to help

[dennismi]

hmmm #171 and #172 has some good ideas ... but not quite sure if they are completely similar .. :)

[pmiossec]

Hi,
In fact the issue #172 is the way I do to manage a tfs branch into git (it works and I think that's the things a git-tfs command should do if you want to add branch support)

and the issue #171 is more an issue that prevent me to use git extension with git-tfs :( not a fondamental problem with git-tfs branch support (at least, that's note quite linked...)

[ismails]

It would be great if git-tfs can help me merge TFS branches. I will also vote for this feature.

[pmiossec]

I did a pull request for that : #232

[pmiossec]

@LievaartR @CoreyKaylor @rbanks54 @jimitndiaye @mika76 @dennismi
Perhaps you should try and give a feedback to #232 and #172 and try the custom version of git-tfs supporting branches : https://github.com/downloads/pmiossec/git-tfs/git-tfs-with-branches.zip

[LievaartR]

Hi,

Thanks for working on this!

I copied the binaries into the git-core directory, but ´git tfs help´ starts by complaining it can not find any exported types in GitTfs.Vs2008. I think I read in one of the issues that you could not build for 2008, so that could be an explanation.

The I cloned your repository and tried to build with

msbuild GitTfs.sln

But that fails as well, the first error is:

D:\tmp\git-tfs.nuget\nuget.targets(86,9): error MSB4067: The element beneath element is unrecognized.
GitTfs\GitTfs.csproj : Solution file warning MSB4122: Scanning project dependencies for project "GitTfs\GitTfs.csproj" failed. The element beneath element is unrecognized. D:\tmp\git-tfs.nuget\nuget.targets

The environment is Windows Xp, Developer Studio 2008, .net 4 is installed, but development is only done for 3.5

I have not looked into the error yet, I guess there is some dependency I am missing, or something I am doing wrong.
But it is Sunday evening, 21:00 and time to call it a day. I´ll try to spend some more time on it in a day or 2.

In the mean time, if anybody has any ideas as to what I am missing or doing wrong, please
let me know.

Regards,

Rob Lievaart.

[sc68cal]

Do you have nuget installed?

[pmiossec]

You're right, the version of git-tfs in the zip file was not supporting TFS 2008 and TFS 2012.
But after that, I succeed in compiling for these 2 versions.
Unfortunately, I had problem with my computer and it was re-installed by the sysadmins.
I will try to re-install the Team Explorer for these versions to be able to compile it...

For your problem, it effectively should be a problem with a lack of nuget...

[worldspawn]

If you are building from source it contains .nuget. Perhaps nuget restore doesn't work/is not supported in pre VS2010. The error indicates it was processing nuget.targets and found something it didn't like, not that it couldn't find the file.

http://stackoverflow.com/questions/3920755/nupack-nuget-on-visual-studio-2008

According to that answer nuget is not supported in vs 2008. Although it's unclear if that extends to the auto-package restore feature which should just be dependent on the version of msbuild being used.

This is in the VS2008.csproj -

<Import Project="$(SolutionDir)\.nuget\nuget.targets" />

Which might be an error?

[LievaartR]

Well, I don't have nuget installed at all, so that explains why it won't build.
I'll install it and have another go, but that will probably be tomorrow.

[LievaartR]

OK, the problem above was caused by running msbuild 3.5, instead of 4.0. The 3.5 version is in my path, if I explicitely run 4.0 it gets a lot further, but it starts complaining about not being able to find some 4.0 libraries. So I guess I have a problem with my 4.0 build setup. I'll let you know once I figured it out.

[LievaartR]

OK, built the tfs_branching branch, currently cloning a repository from tfs (rather large). When done I will try the init-branch all and see what happens.

[pmiossec]

Git tfs init-branch all do not work with tfs 2008 :-( you will have to do all the branches one by one...

[LievaartR]

We should move to 2010 soon (someday).

When doing it manually, it complains I have to set the --parent-branch because 2008 does not support the required functions. But what do I have to enter there?

It allways complains that it cannot find it in the git repository, and that i have to init it first.
I have tried

• the complete tfs name of the parent pranch, the name
• master
• remotes/tfs/default

I also tried to init the MAIN branch, but I keep getting the same complaint.

We have one branch named MAIN that is the master, and all other branches
are direct or indirect descendents from MAIN.

I don't know how obvious the problem is, so below is the command I used and the (debug) result:

I ran this right after a fresh clone, nothing was done on the repository.
The repository contains 2 branches, master and remotes/tfs/default

git tfs init-branch -d --parent-branch=master $/MyProject/MAIN

git command: Starting process: git config --list
git command time: [00:00:00.7343609] config --list
=> Working on TFS branch : $/MyProject/MAIN
git command: Starting process: git check-ref-format --branch MAIN
git command time: [00:00:00.0156247] check-ref-format --branch MAIN
The name of the local branch will be : MAIN
Git local branch will be :MAIN
TFS 2008 Compatible mode!
Sep.Git.Tfs.Core.GitTfsException: error: The Tfs parent branch 'master' can not be found in the Git repository
Please init it before...
---> Sep.Git.Tfs.Core.GitTfsException: error: The Tfs parent branch 'master' can not be found in the Git repository
Please init it before...

at Sep.Git.Tfs.Commands.InitBranch.CreateBranch(IGitTfsRemote defaultRemote, String tfsRepositoryPath, IEnumerable1 allRemotes, String tfsRepositoryPathParentBranch) in d:\tmp\git-tfs\GitTfs\Commands\InitBranch.cs:line 108 at Sep.Git.Tfs.Commands.InitBranch.Run(String argument) in d:\tmp\git-tfs\GitTfs\Commands\InitBranch.cs:line 64 --- End of inner exception stack trace --- at Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList1 args) in d:\tmp\git-tfs\GitTfs\Util\GitTfsCommandRunner.cs:line 35
at Sep.Git.Tfs.GitTfs.Main(GitTfsCommand command, IList1 unparsedArgs) in d:\tmp\git-tfs\GitTfs\GitTfs.cs:line 58 at Sep.Git.Tfs.GitTfs.Run(IList1 args) in d:\tmp\git-tfs\GitTfs\GitTfs.cs:line 42
at Sep.Git.Tfs.Program.Main(String[] args) in d:\tmp\git-tfs\GitTfs\Program.cs:line 24
error: The Tfs parent branch 'master' can not be found in the Git repository
Please init it before...

[pmiossec]

in fact, the --parent-branch parameter is the TFS branch name so you should use a command like that :
git tfs init-branch -d --parent-branch=$/MyProject/MAIN $/MyProject/MyBranch

edit: I have modified the pull request to display a better command line help!

[LievaartR]

That gives me the same result, for completeness I included the command and the output at the bottom..

It feels like it is trying the find the local name for the TFS branch $/MyProject/MAIN, and it can't, but I
never actually told it what my main branch was.

Could it be that I cloned $/MyProject instead of $/MyProject/MAIN ? So I have the whole
project instead of just the main branch?

---

git tfs init-branch -d --parent-branch=$/MyProject/MAIN $/MyProject/DEV/M8138

git command: Starting process: git config --list
git command time: [00:00:01.4530971] config --list
=> Working on TFS branch : $/MyProject/DEV/M8138
git command: Starting process: git check-ref-format --branch M8138
git command time: [00:00:00] check-ref-format --branch M8138
The name of the local branch will be : M8138
Git local branch will be :M8138
TFS 2008 Compatible mode!
Sep.Git.Tfs.Core.GitTfsException: error: The Tfs parent branch '$/MyProject/MAIN' can not be found in the Git repository
Please init it before...
---> Sep.Git.Tfs.Core.GitTfsException: error: The Tfs parent branch '$/MyProject/MAIN' can not be found in the Git repository
Please init it before...

at Sep.Git.Tfs.Commands.InitBranch.CreateBranch(IGitTfsRemote defaultRemote,String tfsRepositoryPath, IEnumerable1 allRemotes, String tfsRepositoryPathParentBranch) in d:\tmp\git-tfs\GitTfs\Commands\InitBranch.cs:line 108 at Sep.Git.Tfs.Commands.InitBranch.Run(String argument) in d:\tmp\git-tfs\GitTfs\Commands\InitBranch.cs:line 64 --- End of inner exception stack trace --- at Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList1 args) in d:\tmp\git-tfs\GitTfs\Util\GitTfsCommandRunner.cs:line 35
at Sep.Git.Tfs.GitTfs.Main(GitTfsCommand command, IList1 unparsedArgs) in d:\tmp\git-tfs\GitTfs\GitTfs.cs:line 58 at Sep.Git.Tfs.GitTfs.Run(IList1 args) in d:\tmp\git-tfs\GitTfs\GitTfs.cs:line 42
at Sep.Git.Tfs.Program.Main(String[] args) in d:\tmp\git-tfs\GitTfs\Program.cs:line 24
error: The Tfs parent branch '$/Aspider.Ews/MAIN' can not be found in the Git repository
Please init it before...

[pmiossec]

Could it be that I cloned $/MyProject instead of $/MyProject/MAIN ? So I have the whole project instead of just the main branch?
Yes it come from here!
In fact in the first clone, you should clone only the trunk and after use 'init-branch' to clone the other branches!

[LievaartR]

OK, thanks.

I'll try that. but that will probably this evening, got to go do some stuff my boss is paying me for :-(

[LievaartR]

It worked!

So far all I did was clone one repository and try to get a few branches. I did run into some issues:

1. It does not seem to detect merges. I do not know if it should do that in this version or if this is
   a 2008 thing.
2. I tried to init a branch that was allready deleted and it was not happy that it did not exist in the
   current version.
3. Branch naming, I did not see any way to name the branches, git-tfs just takes the last part of
   the name in TFS. This iis not allways wat you want, especially since some of our branch names
   are not valid git branch names. In that case I get an error. An option to specify the git name to use
   for the branch woud be nice.

Thanks,

Rob.

[pmiossec]

It worked!
Ok :) You are the first one to test with a TFS2008!

1. No it doesn't support that at the moment. It was a problem I didn't thought about :( Someone, @tamasarpad, inform me about the problem 2 days ago and gave me a way to solve that (I haven't tested yet) but that's not compatible with TFS2008 APIs. So, it will not be disponible with TFS2008!

2. I don't really understand the case. You had a TFS branch and it was merged in the trunk and then deleted? And you still want to create it in TFS? It will surely be resolved by the solution described in 1) but still not with TFS2008 :(

3. No, there is no way to name the branch. In the first version, I provide this feature but remove it when I implemented "init-branch all" feature because I found a way to convert a TFS branch name to a git branch name and then have no names errors. I thought it was better to have nearly the same name for the TFS branch and the git branch names! Perhaps I could put back this feature. I will try and see!

But, you said that you get errors? What was your TFS branch name?

[tamasarpad]

I also had problems with branch names that contains dot.
Made a simple modification in configLineRegex, to allow it to read tfs remotes with dot in their names:

- private static readonly Regex configLineRegex = new Regex("^tfs-remote\\.(?<id>[^.]+)\\.(?<key>[^.=]+)=(?<value>.*)$");
+ private static readonly Regex configLineRegex = new Regex("^tfs-remote\\.(?<id>.+)\\.(?<key>[^.=]+)=(?<value>.*)$");

[pmiossec]

@tamasarpad could you do a pull request (not in my repository but in git-tfs official one) for this problem of dot in git branch name (which is permitted by git!)?

[pmiossec]

@LievaartR @tamasarpad @worldspawn New version!

• New way to find root changeset (on TFS2010 and 2012) to handle branches merges in trunk by @tamasarpad !!
• You could know suggest a name for a branch cloned
• To clone all the branches, now, it's an option 'git init-branch --all' (still not for TFS2008 :( )

go test! ;)

[tamasarpad]

It work's fine for me. Thanks!

[pmiossec]

@tamasarpad finally, I've included your modification to managed branches with '.' in their names in the pull request otherwise it breaks the init-branch.

We should be OK for all!

Do you think we should push maintainers to merge the pull request or there is other problems to fix?
I can't see any problem remaining at the moment...

[sc68cal]

Looking very nice. 👍 for merge. @spraints ?

[tamasarpad]

Thanks Philippe!
I use it daily (with the fixes) and haven't found any problem yet (cloned our tfs repos/branches to several environments).

[pmiossec]

You use it more than me ;) I use it nearly just in a test repository...

I just want that git was more used in my team and I did it because it was a lack of git-tfs and I didn't want it could be a problem and a reproach against its adoption...

[pmiossec]

new version. Someone could test it? Expected to work like before....

[pmiossec]

@LievaartR @tamasarpad @worldspawn Noone to test this version?

I think that's the last right line toward the merge ;) In any case, I have no more things in my todo list...

[tamasarpad]

Sorry I didn't have the time to test yet.
Tried it out today with the latest changes (pmiossec@4f7c737) and unfortunately I got this exception many times:
Unhandled Exception: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at LibGit2Sharp.Core.NativeMethods.git_repository_free(IntPtr repo)
at LibGit2Sharp.Core.Proxy.git_repository_free(IntPtr repo)
at LibGit2Sharp.Core.Handles.RepositorySafeHandle.ReleaseHandle()
at System.Runtime.InteropServices.SafeHandle.InternalDispose()
at System.Runtime.InteropServices.SafeHandle.Dispose(Boolean disposing)
at LibGit2Sharp.Core.Handles.SafeHandleBase.Dispose(Boolean disposing)
at System.Runtime.InteropServices.SafeHandle.Dispose()
at LibGit2Sharp.Core.Handles.SafeHandleExtensions.SafeDispose(SafeHandleBase handle)
at LibGit2Sharp.Core.Handles.SafeHandleExtensions.SafeDispose(IDisposable disposable)
at LibGit2Sharp.Repository.Dispose(Boolean disposing)
at LibGit2Sharp.Repository.Finalize()

[nulltoken]

Hmmm. That is unexpected. Is there any chance you could provide me with a scoped repro case which doesn't involve TFS? I'll be happy to troubleshoot this from the LibGit2Sharp side.

[pmiossec]

@tamasarpad
The last commit of the new version you used, update the libgit2sharp version used from the stable one (master branch) to the 'in development' one (vNext). The problem should come from that. I let you see if you could solve the problem with @nulltoken because I don't have the problem...

That thing appart, I push a new version which correct a bad use of the method repo.Refs.IsValidName() which prevent a good functionnment of git-tfs (but which should not correct your problem :( ). Try nevertheless, perhaps the problem come for there...

[pmiossec]

@tamasarpad @nulltoken in fact, I have the problem! But I don't know exactly where it come from and how to reproduce it without tfs.
And it's difficult to understand the problem because it fail during the Dispose() but not everytime (only when you do some things and I don't know which ones...)

[pmiossec]

After quite some tests, I don't have ideas where the problem come from :(
On my work project, where we use TFS2012, I have the problem when I fetch tfs commits (and when I rcheckin, because i do a fetch again) but on the codeplex project I use for my tests, I never have the problem (but I don't know the TFS version). Not easy to understand.....

@tamasarpad Which version of TFS do you work with?

Will try to have a debug trace tomorrow...

[nulltoken]

Might be related to libgit2/libgit2sharp#249

[tamasarpad]

I got the exceptions with TFS 2010, but I don't think has anything to do with TFS. I also tried to reproduce it in an isolated environment but I haven't succeeded yet. It gives the exceptions randomly...

[pmiossec]

@nulltoken tested the solution described in libgit2/libgit2sharp#249 and it works fine! No more problem.

Waiting for the merge, now...

[nulltoken]

Indeed @markus-olsson totally saved the day! This will be merged by tomorrow.

On a less positive note, this also means that you're missing a Dispose() call somewhere...

[pmiossec]

Indeed @markus-olsson totally saved the day! This will be merged by tomorrow.
Yeah! Congrats to him!!!

On a less positive note, this also means that you're missing a Dispose() call somewhere...
I will let the main developpers of git-tfs find where it is ;)

[niik]

Thanks for the love 😊 Man, if this blows up or creates an issue elsewhere I'm gonna die of shame :)

[nulltoken]

if this blows up or creates an issue elsewhere I'm gonna die of shame :)

Yes, you will. Then, you'll rise and fix all the things!

[nulltoken]

@markus-olsson 's libgit2/libgit2sharp#249 has been merged in vNext

/cc @pmiossec

[pmiossec]

@nulltoken @markus-olsson Ok. Thanks!

[pmiossec]

@tamasarpad I pushed a new version that should correct the problem.
Give some feedback because, in my opinion, we are ready to merge the pull request...

[tamasarpad]

It works fine again for me! Thanks!

[pmiossec]

@spraints do we close this issue now that the feature was merged?

[sparkskapil]

I have found an issue while migrating our TFS Project to GIT.
Some of the changesets on Development Branch of TFVC are not present on Development of GITVC
Those missing changesets got migrated on master branch.
I think this should not have happened.