gitcc does not handle moved directories which causes files to disappear. about git-cc

[charleso]

My answer for this is the same as #22. Git doesn't track renames, and it especially doesn't track directories. Now, of course, there is certainly something lost if you move a directory in Git, and then run 'gitcc checkin'. This is a problem, but frankly IMHO not one I think is worth solving. In this case I rename each file manually in Clearcase, but I'm manually adding/removing directories. Anything else would be, um, complicated and probably quite buggy.

[nesqi]

First of all I'm only talking about gitcc rebase here.

This is not the same problem as in #22.

Let me walk you through an example.

In ClearCase a user moves a directory from place A to B. That means that the clearcase command lshistory will report two different "checkindirectory version"-events. One event where the directory is removed from A, and one where the directory is added to B.

For both of these events gitcc will run the command "cleartool diff -diff_format -pred

" to see what have changed. For directory A the output of the command will look like

-----[ deleted ]-----
< moveddirectory/ 2001-08-22 someuser

what gitcc now does is to run "git rm -r moveddirectory" to remove the directory and all it's contents. This might be ok depending on what is done with the other message. (The cache is not cleared of the files that are now removed.)

For directory B the output of the command looks like

-----[ added ]-----

moveddirectory/ 2001-08-22 someuser

How gitcc interprets this information is the core of the problem. Instead of recreating the contents of the directory that was MOVED in clearcase. gitcc only thinks that a new directory was created. This means that all the files that are in the directory in ClearCase are missing in the git-repo. If the files are changed later in the ClearCase history they we be updated but all their history will be lost as git does not track renaming for files that are removed in one commit and added in another commit.

[charleso]

Apologies, you're quite right - I don't handle that. Reopened.

    if not exists(cc_added) or isdir(cc_added) or added in files:
        continue

That's the offending line. You will have to get a list of files at at that version and 'get' each sub file/folder.

[charleso]

Keep in mind, given the way that Git doesn't care about renames, we still don't need to think about the 'rename' any differently. Just get the contents of all the files and you're done...

[nesqi]

The rename has nothing to do with the name of the file but rather a place to put the file until it reappears in another commit. Something like, all removed files and directories are moved to

GIT_ROOT/.unlinked/

this way it would be easy to find the files that should be reinserted and there is a way to guarantee that the file is not deleted in one commit and re-added in another commit in which case it will loose it's history.

[charleso]

I don't understand how moving files out to that directory is going to keep a files history. What would you do with that folder once is required again? Move it back into the working directory, run 'git add folder' and commit. How is that any different from just getting the contents of each file in Clearcase and adding it directly? As far as git is concerned it's exactly the same thing. Git just doesn't care...

[charleso]

On the other hand if you're saying this is a way to easy get back the entire folder, then that may be a neat hack. Ideally you can recreate the entire thing by running a combination of 'cleartool ls' and 'cleartool get', but from memory 'ls' isn't that great at looking at old revisions (although I might be making that up). Can you easily tell what the 'ID' of a folder is in git? I'm assuming you can't go basing it just on the folder 'name', which of course may change.

Apologies if I'm misreading all of your comments. It's been too long since I've done any Clearcase stuff. Maybe I should just shut up... :)

[nesqi]

Another example =)

Here are two identical ways to move a file in a git repo.

git add a
git commit -am "added a"
git mv a b
git commit -am "moved a->b"

and

git add a
git commit -am "added a"
cp a b
git rm a
git add b
git commit -am "moved a->b"

Here is a third example which is NOT equivalent. In this example the files history is lost.

git add a
git commit -am "added a"
cp a b
git rm a
git commit -am "removed a"
git add b
git commit -am "added b"

this third version is what happens in gitcc sometimes and history is lost for files.

[nesqi]

The OID in my comment above

GIT_ROOT/.unlinked/

is the OID that every file-element has in ClearCase.

http://stackoverflow.com/questions/6247134/how-do-i-determine-if-a-clearcase-element-has-been-moved-renamed

[charleso]

So, a few points.

Firstly I'd be a little careful with saying Git has "lost" your history. Sure, you can't easily query it now. But with your example you could run:

git diff --name-status -M HEAD^^

Or

git reset HEAD^
git add b
git commit --amend

And as if by magic you now have a 'rename' again. The information is still there, you just have to look harder to see it. As mentioned previously, one solution for this for a once-off clearcase -> git conversion would be to write a simple script to look for these cases in the history and to rewrite it with 'git filter-branch'.

Now, that said, it would lovely if git-cc did the right thing here, and so other people can reap the benefits. That is going to be more work. If/when someone gets around to doing it, and they want to try your approach, one suggestion is that you (really) don't need to copy files into a subdirectory of Git.

Every folder in Git has a unique hash based on the recursive child filenames and contents. You can see it by doing:

git ls-tree HEAD some/folder

You could then, hypothetically, have a mapping file like:

<oid1> <tree1>
<oid2> <tree2>

When you need to resurrect that folder again after a rename simply run:

git read-tree -u --prefix=new/folder <tree1>

Git is already storing this information for you, you just need a way to retrieve it.

All that said, my preference would always be to make the solution more generic. Ideally you could run something like:

cleartool get -to /gitrepo/newfolder new/folder@/main/brancha

And have it output the contents of the entire folder at a given branch. That would save you having to worry about keeping track of the OIDs and renames, which is complicated. I can only assume this won't actually work. :(

Even doing all of this, you still have the problem of 'losing history' between commits. What might be nice is to collect the OID of each folder, and 'merge' the (potential) commits if any two Changets (see rebase.py) have a matching OID, regardless of the comment message. I think you might be able to do that with "%On" in lshistory:

http://www.ipnom.com/ClearCase-Commands/fmt_ccase.html

This avoids the problem above because you could simply just move the folder on the filesystem and avoid all of complicated mess above.

I hope some of this helps.