OutOfMemoryException about git-tfs

[spraints]

yeah, you can do git tfs fetch, and it'll continue.

The crash was probably from a large file, so it may crash again. If it
does and you're not on the latest version from github, could you try
using the latest version from github? JamesDunne submitted a patch
that changes how files are handled when fetching from TFS, and it
might resolve this issue.

On Fri, May 6, 2011 at 12:39 PM, staxmanade
[email protected]
wrote:

Started the process of pulling changes out of a good sized tfs repository and received an OutOfMemoryException.
It was at about TFS change 3000 of 7000 and the entire git repo (size on disk) had reached about 130 meg.

Is there a way to have it pick-up again where it was at?

Reply to this email directly or view it on GitHub:
https://github.com/spraints/git-tfs/issues/42

[staxmanade]

I first tried the latest when I started this project (of exporting our TFS projects) - but I couldn't get the project to build, so I just used the latest setup in your "downloads".

I'll try again to get the latest.

And Yes it does have to do with a large file 1.2gig in this case. (Found out on a different project w/ a different file. but seemed to be the issue.)

[staxmanade]

I pulled the latest again and tried to build. I'm getting build errors in the GitTfs.VS2008 project - I removed that and re-build. I reconfigured my PATH to point the /bin/debug folder and ran another test.

Still seeing the OutOfMemoryException

[staxmanade]

This may be more of a git.Sharp issue but here's the whole stack trace.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.OutOf
MemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at GitSharp.Core.ObjectWriter.WriteObject(ObjectType type, Int64 len, Stream input, Boolean store) in d:\dev\buildAge
nt\work\5e044ae681b5c534\GitSharp.Core\ObjectWriter.cs:line 251
at GitSharp.Core.ObjectWriter.WriteBlob(Int64 len, Stream input) in d:\dev\buildAgent\work\5e044ae681b5c534\GitSharp.
Core\ObjectWriter.cs:line 121
at Sep.Git.Tfs.Core.GitRepository.HashAndInsertObject(Stream file, Int64 length) in E:\Code_staxmanade\github\git-tfs
\GitTfs\Core\GitRepository.cs:line 364
at Sep.Git.Tfs.Core.GitIndexInfo.Update(String mode, String path, Stream stream, Int64 length) in E:\Code_staxmanade
github\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 62
at Sep.Git.Tfs.Core.TfsChangeset.Update(IChange change, String pathInGitRepo, GitIndexInfo index, IDictionary2 initi alTree) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\TfsChangeset.cs:line 116 at Sep.Git.Tfs.Core.TfsChangeset.Apply(IChange change, GitIndexInfo index, IDictionary2 initialTree) in E:\Code_stax
manade\github\git-tfs\GitTfs\Core\TfsChangeset.cs:line 55
at Sep.Git.Tfs.Core.TfsChangeset.Apply(String lastCommit, GitIndexInfo index) in E:\Code_staxmanade\github\git-tfs\Gi
tTfs\Core\TfsChangeset.cs:line 31
at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass6.b__4(GitIndexInfo index) in E:\Code_staxmanade\github\git-
tfs\GitTfs\Core\GitTfsRemote.cs:line 251
at Sep.Git.Tfs.Core.GitIndexInfo.Do(TextWriter stdin, IGitRepository repository, Action1 action) in E:\Code_staxmana de\github\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 21 at Sep.Git.Tfs.Core.GitIndexInfo.<>c__DisplayClass1.<Do>b__0(TextWriter stdin) in E:\Code_staxmanade\github\git-tfs\G itTfs\Core\GitIndexInfo.cs:line 12 at Sep.Git.Tfs.Core.GitHelpers.<>c__DisplayClassb.<CommandInputPipe>b__a() in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitHelpers.cs:line 160 at Sep.Git.Tfs.Core.GitHelpers.Time(String[] command, Action action) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitHelpers.cs:line 182 at Sep.Git.Tfs.Core.GitHelpers.CommandInputPipe(Action1 action, String[] command) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitHelpers.cs:line 156
at Sep.Git.Tfs.Core.GitIndexInfo.Do(IGitRepository repository, Action1 indexAction) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 12 at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass6.<Apply>b__3() in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 251 at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass1a.<WithTemporaryIndex>b__19() in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 344 at Sep.Git.Tfs.Core.GitTfsRemote.WithTemporaryEnvironment(Action action, IDictionary2 newEnvironment) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 354
at Sep.Git.Tfs.Core.GitTfsRemote.WithTemporaryIndex(Action action) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 341
at Sep.Git.Tfs.Core.GitTfsRemote.Apply(String lastCommit, ITfsChangeset changeset) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 250
at Sep.Git.Tfs.Core.GitTfsRemote.Fetch(Dictionary2 mergeInfo) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 138 at Sep.Git.Tfs.Core.Ext.Fetch(IGitTfsRemote remote) in E:\Code_staxmanade\github\git-tfs\GitTfs\Core\IGitTfsRemote.cs:line 36 at Sep.Git.Tfs.Commands.Fetch.DoFetch(IGitTfsRemote remote) in E:\Code_staxmanade\github\git-tfs\GitTfs\Commands\Fetch.cs:line 64 at Sep.Git.Tfs.Commands.Fetch.Run(String[] args) in E:\Code_staxmanade\github\git-tfs\GitTfs\Commands\Fetch.cs:line 57 at Sep.Git.Tfs.Commands.Fetch.Run() in E:\Code_staxmanade\github\git-tfs\GitTfs\Commands\Fetch.cs:line 49 at Sep.Git.Tfs.Commands.Clone.Run(String tfsUrl, String tfsRepositoryPath, String gitRepositoryPath) in E:\Code_staxmanade\github\git-tfs\GitTfs\Commands\Clone.cs:line 39 at Sep.Git.Tfs.Commands.Clone.Run(String tfsUrl, String tfsRepositoryPath) in E:\Code_staxmanade\github\git-tfs\GitTfs\Commands\Clone.cs:line 32 --- End of inner exception stack trace --- at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner) at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks) at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) at Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList1 args) in E:\Code_staxmanade\github\git-tfs\GitTfs\Util\GitTfsCommandRunner.cs:line 36
at Sep.Git.Tfs.GitTfs.Main(GitTfsCommand command, IList1 unparsedArgs) in E:\Code_staxmanade\github\git-tfs\GitTfs\GitTfs.cs:line 56 at Sep.Git.Tfs.GitTfs.Run(IList1 args) in E:\Code_staxmanade\github\git-tfs\GitTfs\GitTfs.cs:line 39
at Sep.Git.Tfs.Program.Main(String[] args) in E:\Code_staxmanade\github\git-tfs\GitTfs\Program.cs:line 23
Exception of type 'System.OutOfMemoryException' was thrown.

[spraints]

It is a gitsharp issue. I think the latest gitsharp fixes this, but
there wasn't a release that included it. i had considered using
libgit2, but haven't had a chance to try it out.

[cjackson]

I'm also getting an error when trying to clone my TFS repository. Here's my stack trace, using version 0.12.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
at System.IO.MemoryStream.EnsureCapacity(Int32 value)
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at GitSharp.Core.ObjectWriter.WriteObject(ObjectType type, Int64 len, Stream
input, Boolean store) in d:\dev\buildAgent\work\5e044ae681b5c534\GitSharp.Core\ObjectWriter.cs:line 251
at Sep.Git.Tfs.Core.GitRepository.HashAndInsertObject(Stream file, Int64 length) in D:\work\git-tfs\GitTfs\Core\GitRepository.cs:line 437
at Sep.Git.Tfs.Core.GitIndexInfo.Update(String mode, String path, Stream stream, Int64 length) in D:\work\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 62
at Sep.Git.Tfs.Core.TfsChangeset.Update(IChange change, String pathInGitRepo, GitIndexInfo index, IDictionary2 initialTree) in D:\work\git-tfs\GitTfs\Core\TfsChangeset.cs:line 114 at Sep.Git.Tfs.Core.TfsChangeset.Apply(IChange change, GitIndexInfo index, IDictionary2 initialTree) in D:\work\git-tfs\GitTfs\Core\TfsChangeset.cs:line 55
at Sep.Git.Tfs.Core.TfsChangeset.Apply(String lastCommit, GitIndexInfo index) in D:\work\git-tfs\GitTfs\Core\TfsChangeset.cs:line 31
at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass6.b__4(GitIndexInfo
index) in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 299
at Sep.Git.Tfs.Core.GitIndexInfo.Do(TextWriter stdin, IGitRepository repository, Action1 action) in D:\work\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 21 at Sep.Git.Tfs.Core.GitIndexInfo.<>c__DisplayClass1.<Do>b__0(TextWriter stdin) in D:\work\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 12 at Sep.Git.Tfs.Core.GitHelpers.<>c__DisplayClassb.<CommandInputPipe>b__a() in D:\work\git-tfs\GitTfs\Core\GitHelpers.cs:line 160 at Sep.Git.Tfs.Core.GitHelpers.Time(String[] command, Action action) in D:\work\git-tfs\GitTfs\Core\GitHelpers.cs:line 182 at Sep.Git.Tfs.Core.GitHelpers.CommandInputPipe(Action1 action, String[] command) in D:\work\git-tfs\GitTfs\Core\GitHelpers.cs:line 156
at Sep.Git.Tfs.Core.GitIndexInfo.Do(IGitRepository repository, Action1 indexAction) in D:\work\git-tfs\GitTfs\Core\GitIndexInfo.cs:line 12 at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass6.<Apply>b__3() in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 299 at Sep.Git.Tfs.Core.GitTfsRemote.<>c__DisplayClass1a.<WithTemporaryIndex>b__19() in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 392 at Sep.Git.Tfs.Core.GitTfsRemote.WithTemporaryEnvironment(Action action, IDictionary2 newEnvironment) in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 402
at Sep.Git.Tfs.Core.GitTfsRemote.WithTemporaryIndex(Action action) in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 389
at Sep.Git.Tfs.Core.GitTfsRemote.Apply(String lastCommit, ITfsChangeset changeset) in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 298
at Sep.Git.Tfs.Core.GitTfsRemote.FetchWithMerge(Int64 mergeChangesetId, String[] parentCommitsHashes) in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 162
at Sep.Git.Tfs.Core.GitTfsRemote.Fetch() in D:\work\git-tfs\GitTfs\Core\GitTfsRemote.cs:line 154
at Sep.Git.Tfs.Commands.Fetch.DoFetch(IGitTfsRemote remote) in D:\work\git-tfs\GitTfs\Commands\Fetch.cs:line 70
at Sep.Git.Tfs.Commands.Fetch.Run(String[] args) in D:\work\git-tfs\GitTfs\Commands\Fetch.cs:line 57
at Sep.Git.Tfs.Commands.Fetch.Run() in D:\work\git-tfs\GitTfs\Commands\Fetch.cs:line 49
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Sep.Git.Tfs.Util.GitTfsCommandRunner.Run(GitTfsCommand command, IList1 args) in D:\work\git-tfs\GitTfs\Util\GitTfsCommandRunner.cs:line 36 at Sep.Git.Tfs.GitTfs.Main(GitTfsCommand command, IList1 unparsedArgs) in D:\work\git-tfs\GitTfs\GitTfs.cs:line 56
at Sep.Git.Tfs.GitTfs.Run(IList`1 args) in D:\work\git-tfs\GitTfs\GitTfs.cs:line 39
at Sep.Git.Tfs.Program.Main(String[] args) in D:\work\git-tfs\GitTfs\Program.cs:line 23
Exception of type 'System.OutOfMemoryException' was thrown.

[ahri]

I compiled the latest git-tfs and GitSharp, and now the OOM seems to have gone away (whereas previously it would happen every time for me on our stupid repo :))

I do note that doing the full clone takes a long time, and actually resulted in the following timeout:

C18045 = ddf7a18a2afcac1635e8b5d63f4264f3a0e3f1f6
TF15004: The download request signature has expired.
TF15004: The download request signature has expired.
PackFile has not been properly disposed: {C:\Users\adam.piper\TFS\project\.git\objects\pack\pack-dc346b490a7dded3f617f178f36eedcf24e22a6b.pack}/{C:\Users\adam.piper\TFS\project\.git\objects\pack\pack-dc346b490a7dded3f617f178f36eedcf24e22a6b.idx}

Does this warrant a separate bug? Also, do you want copies of the files I compiled?

[sc68cal]

Going to close this since v0.14.0 is the last version that incorporates GitSharp.