wangweiwei / gclient Goto Github PK

Every once or so, we get:
svn: Server sent unexpected return value (502 Bad Gateway) in response to
PROPFIND request for '/svn/trunk/deps/third_party/icu42'
Error: failed to run command: svn update

This might be ok for human users, but when gclient is used in bots (such as 
in build.webkit.org), this creates a lot of noise.

See issue in bugs.webkit.org: https://bugs.webkit.org/show_bug.cgi?
id=33655

"to_info" can be {}, so accessing to_info[...] causes a KeyError. It would be 
good to add a check and report a decent error message in this case. I'm not 
familiar enough with this part of the code to provide a decent message.

Suggested addition after line 170:
      if to_info == {}:
          raise gclient_utils.Error("...error message...");

When an entry has been removed from the deps file, running a sync will 
report "%s is no longer part of this client. It is recommended that you 
manually remove it."

It'd be nice to have the option to have gclient delete the directory on its 
own. Then scripts that check out projects in order to do something else 
with them (e.g., build a tarball of the tree) can keep themselves cleaned 
up.

Ideal would be to only delete the outdated directory if it has no pending 
changes, but for a first step it might be all right to just wipe it out, 
with appropriate warnings in the option's help.

What steps will reproduce the problem?
1. gclient sync
2. disconnect network cable.

gclient doesn't cache in .gclient_entries the fact that a svn run returned 
non-zero. So the checkout is left in an inconsistent state until --force is 
used.

Since svn failure seems to happen quite often, users are left wondering how 
to fix this.

What steps will reproduce the problem?
1. export LC_ALL=pt_BR
2. gclient sync
3.

What is the expected output?
gclient should sync the repository

What do you see instead?

________ found .git directory; skipping src
Traceback (most recent call last):
  File "gclient.py", line 937, in <module>
    result = Main(sys.argv)
  File "gclient.py", line 933, in Main
    return DispatchCommand(command, options, args)
  File "gclient.py", line 897, in DispatchCommand
    return command_map[command](options, args)
  File "gclient.py", line 849, in DoUpdate
    return update_all(client, options, args)
  File "gclient.py", line 725, in UpdateAll
    deps = get_all_deps(client, entries)
  File "gclient.py", line 619, in GetAllDeps
    url = info["Repository Root"] + url
KeyError: 'Repository Root'


What version of the product are you using?
SVN checkout of revision 25

On what operating system?
Debian GNU/Linux

I've attached a patch that should fix this issue.

On some localized versions of linux, 'svn info' doesn't return English 
text.  When gclient tries to parse the output, it fails.

One workaround is to set LANG=C before running svn info.  Maybe we should 
do this for all svn commands?

What steps will reproduce the problem?
1. Create changelist, get review, make changes.
2. Upload new changelist via `gcl upload <changelist>` (successful)
3. Attempt to commit changelist via `gcl commit <changelist>` ...fail.

What is the expected output? What do you see instead?
Issue should be updated and changelist should be committed. Instead I get 
an error that I can not access the url, which I assume it is trying to use 
to read the changelist description.

What version of the product are you using? On what operating system?
gclient v1.1.1
Ubuntu 9.04 jaunty (64bit)

Please provide any additional information below.
After making appropriate changes to 
http://codereview.appspot.com/124048/show, I attempt to commit the change 
and it gives me an error that it cant access the description url:

$ /home/nmaxwell/depot_tools/gcl commit makeself
Email [[email protected]]: 
Password for [email protected]: 
Error accessing url /124048/description

Im not sure why this is happening. For background, I recently suffered from 
the bug in (http://code.google.com/p/rietveld/issues/detail?id=158), which 
was merged with issue #156, but the ownership issue has since been fixed it 
appears. Now, the issue clearly shows that I ([email protected]) am 
the owner of the issue, however I can not commit the changelist, though I 
can upload it successfully. Any idea whats happening here? FYI, I am a 
Googler too, [email protected] is my internal email if you discover this 
is due to an internal issue. Thanks. 

What steps will reproduce the problem?
1. LANG=C ./depot_tools/gclient sync

What is the expected output? What do you see instead?
Traceback (most recent call last):
  File "/home/pal/work/gc/chromium/depot_tools/release/gclient.py", line
928, in <module>
    result = Main(sys.argv)
  File "/home/pal/work/gc/chromium/depot_tools/release/gclient.py", line
924, in Main
    return DispatchCommand(command, options, args)
  File "/home/pal/work/gc/chromium/depot_tools/release/gclient.py", line
888, in DispatchCommand
    return command_map[command](options, args)
  File "/home/pal/work/gc/chromium/depot_tools/release/gclient.py", line
833, in DoUpdate
    client = get_client()
  File "/home/pal/work/gc/chromium/depot_tools/release/gclient.py", line
494, in GetClient
    exec(client_source, client)
  File "<string>", line 1

 ^
SyntaxError: invalid syntax


What version of the product are you using? On what operating system?
Linux

Please provide any additional information below.
It happen when .gclient file is CRLF
>% file .gclient
>.gclient: ASCII English text, with CRLF line terminators
Use python universal-newlines mode.

Index: gclient.py
===================================================================
--- gclient.py  (revision 1663)
+++ gclient.py  (working copy)
@@ -488,7 +488,7 @@
     path = next[0]
     client_file = os.path.join(path, CLIENT_FILE)
   client = {}
-  client_fo = open(client_file)
+  client_fo = open(client_file, 'U')
   try:
     client_source = client_fo.read()
     exec(client_source, client)

What steps will reproduce the problem?
1. svn delete a file
2. change its contents
3. svn add the file
4. make change list and gcl upload it

What is the expected output?
Expecting it to change to an update.

What do you see instead?
Issue creation errors: {'data': [u'The submitted file is empty.']}


What version of the product are you using? On what operating system?
vista64

Please provide any additional information below.

What steps will reproduce the problem?
1. open Windows Command Prompt and change to my trunk directory
2. from the prompt, enter "gclient sync"

What is the expected output? What do you see instead?
I expect so see some sort of progress for checking out my project.  Instead 
I see an error ending in "Cannot locate w9xpopen.exe, which is needed for 
Popen to work with your shell or platform."  Note that this is my first 
time using gclient so I have never been successful in running "gclient 
sync".

What version of the product are you using? On what operating system?
Please let me know how to check the version; I downloaded my version from 
http://build.chromium.org/buildbot/archives/depot_tools_win.zip.  I am 
running Microsoft Windows XP, service pack 3.

Please provide any additional information below.

Full error from command prompt:
D:\JD\Chrome\trunk>gclient sync

________ running 'svn checkout http://src.chromium.org/svn/trunk/src src' 
in 'D:
\JD\Chrome\trunk'
Traceback (most recent call last):
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 1091, in ?
    result = Main(sys.argv)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 1087, in Main
    return DispatchCommand(command, options, args)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 1048, in 
DispatchCom
mand
    return command_map[command](options, args)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 964, in 
DoUpdate
    return update_all(client, options, args)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 834, in 
UpdateAll
    r = update_to_url(name, url, client["root_dir"], options, args)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 536, in 
UpdateToURL
    return run_svn(c, run_dir)
  File "D:\JD\Chrome\depot_tools\release\\gclient.py", line 307, in RunSVN
    rv = call(c, cwd=in_directory, shell=(sys.platform == 'win32'))
  File "D:\JD\Chrome\depot_tools\release\python_24\lib\subprocess.py", line 
428,
 in call
    return Popen(*args, **kwargs).wait()
  File "D:\JD\Chrome\depot_tools\release\python_24\lib\subprocess.py", line 
558,
 in __init__
    errread, errwrite)
  File "D:\JD\Chrome\depot_tools\release\python_24\lib\subprocess.py", line 
701,
 in _execute_child
    w9xpopen = self._find_w9xpopen()
  File "D:\JD\Chrome\depot_tools\release\python_24\lib\subprocess.py", line 
663,
 in _find_w9xpopen
    raise RuntimeError("Cannot locate w9xpopen.exe, which is "
RuntimeError: Cannot locate w9xpopen.exe, which is needed for Popen to work 
with
 your shell or platform.

D:\JD\Chrome\trunk>

[What steps will reproduce the problem?]
1. Simply type gclient sync in shell window

[What is the expected output? ]
i can start sync.

[What do you see instead?]
I got the following error when i try to sync the source
checking out latest depot_tools...
svn: OPTIONS of 
'http://src.chromium.org/svn/trunk/depot_tools/release/win': Could not 
resolve hostname `src.chromium.org': The requested name is valid, but no
data of the requested type was found.   (http://src.chromium.org)
The system cannot find the path specified.

[What version of the product are you using? On what operating system?]
gclient from depot_tools.zip
Windows Vista

Windows only issue.

gclient should use a full path when referring to svn.exe

What steps will reproduce the problem?
1. Create this .gclient file:
solutions = [
  { "name"        : "master.tryserver.DEPS",
    "url"         : "svn://chrome-svn.corp.google.com/chrome-
internal/trunk/tools/buildbot/master.tryserver.DEPS",
    "custom_deps" : {
    }
  },
  { "name"        : "master.chromium.DEPS",
    "url"         : "svn://chrome-svn.corp.google.com/chrome-
internal/trunk/tools/buildbot/master.chromium.DEPS",
    "custom_deps" : {
    }
  },
  { "name"        : "slave.DEPS",
    "url"         : "svn://chrome-svn.corp.google.com/chrome-
internal/trunk/tools/buildbot/slave.DEPS",
    "custom_deps" : {
    }
  },
]

2. gclient sync

What is the expected output? What do you see instead?
________ running 'svn update D:\b.pub\slave' in 'D:\b.pub'
At revision 10698.
Error: Solutions have conflicting versions of dependency "pylibs"

What steps will reproduce the problem?
1. commit a list with 1100 files

What is the expected output?
It should work

What do you see instead?
Warning: post-commit hook failed (exit code 2) with output:
Post-commit mail timed out for r37783. Your commit succeeded,
but no mail was sent.

What version of the product are you using? On what operating system?
Vista64

Please provide any additional information below.
The gcl upload also had timeouts and took more than 15 minutes.

sanxiyn on IRC suggested that we use "svn info --xml" instead of "LANG=C 
svn info".  This ensures that the output is machine readable and locale 
independent.

If 'gclient update' fails, the local repository may be left with locks that
have not been released.  It would be nice if gclient supported the 'svn
cleanup' command so that the locking problem could be resolved without the
need to manually run 'svn cleanup' in special directories such as
chromium's third_party.

What steps will reproduce the problem?
1. svn delete *
2. robocopy newfolder currentfolder /s
3. svn add *
4. gcl change newcode
5. gcl upload newcode

What is the expected output? What do you see instead?
should upload to code review site
Got error status from ['svn', 'cat', 'third_party/ffmpeg/patched-ffmpeg-
mt/configure']:

What version of the product are you using? On what operating system?
windows vista 64

Please provide any additional information below.
its a fairly large change list... 1344 files.
many files have been deleted or renamed, which is why I did svn delete 
first.
I'm guessing its the size of the change that caused the issue.

What steps will reproduce the problem?
1. svn rm an empty directory
2. make a CL and upload it

What is the expected output? 
Creation of a code review issue.

What do you see instead?
Issue creation errors: {'data': [u'The submitted file is empty.']}

What version of the product are you using? On what operating system?
gcl v1.1.3 on win7 64bit

Please provide any additional information below.

What steps will reproduce the problem?
1.gclient sync 
2. when run on cmd shell 
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
windows vista sp 1 home premium


Please provide any additional information below.





follwing error  occur during gclient sync on cmd shell


D:\downloads\[chromiumos]>gclient sync
Traceback (most recent call last):
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1183, in ?
    result = Main(sys.argv)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1178, in Main
    return DispatchCommand(command, options, args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1103, in
Dispat
chCommand
    return command_map[command](options, args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1020, in
DoUpda
te
    return client.RunOnDeps('update', args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 675, in
RunOnDe
ps
    scm.RunCommand(command, self._options, args, file_list)
  File "D:\downloads\depot_tools\gclient_scm.py", line 79, in RunCommand
    return getattr(self, command)(options, args, file_list)
  File "D:\downloads\depot_tools\gclient_scm.py", line 131, in update
    cwd=self._root_dir, redirect_stdout=False)
  File "D:\downloads\depot_tools\gclient_scm.py", line 197, in _Run
    sp = subprocess.Popen(cmd, cwd=cwd, stdout=stdout)
  File "D:\downloads\depot_tools\python_bin\lib\subprocess.py", line 558,
in __i
nit__
    errread, errwrite)
  File "D:\downloads\depot_tools\python_bin\lib\subprocess.py", line 722,
in _ex
ecute_child
    startupinfo)
WindowsError: [Errno 2] The system cannot find the file specified

What steps will reproduce the problem?
1.gclient sync 
2. when run on cmd shell 



What version of the product are you using? On what operating system?
windows vista sp 1 home premium
depot_tools download from
 http://src.chromium.org/svn/trunk/tools/depot_tools


Please provide any additional information below.





follwing error  occur during gclient sync on cmd shell


D:\downloads\[chromiumos]>gclient sync
Traceback (most recent call last):
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1183, in ?
    result = Main(sys.argv)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1178, in Main
    return DispatchCommand(command, options, args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1103, in
Dispat
chCommand
    return command_map[command](options, args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 1020, in
DoUpda
te
    return client.RunOnDeps('update', args)
  File "D:\downloads\depot_tools\bootstrap\\..\gclient.py", line 675, in
RunOnDe
ps
    scm.RunCommand(command, self._options, args, file_list)
  File "D:\downloads\depot_tools\gclient_scm.py", line 79, in RunCommand
    return getattr(self, command)(options, args, file_list)
  File "D:\downloads\depot_tools\gclient_scm.py", line 131, in update
    cwd=self._root_dir, redirect_stdout=False)
  File "D:\downloads\depot_tools\gclient_scm.py", line 197, in _Run
    sp = subprocess.Popen(cmd, cwd=cwd, stdout=stdout)
  File "D:\downloads\depot_tools\python_bin\lib\subprocess.py", line 558,
in __i
nit__
    errread, errwrite)
  File "D:\downloads\depot_tools\python_bin\lib\subprocess.py", line 722,
in _ex
ecute_child
    startupinfo)
WindowsError: [Errno 2] The system cannot find the file specified

D:\downloads\[chromiumos]>

What steps will reproduce the problem?
1. svn delete *
2. gcl change / upload

What is the expected output?
make a code review

What do you see instead?
Traceback (most recent call last):
  File "c:\bin\depot_tools\gcl.py", line 1214, in ?
    sys.exit(main())
  File "c:\bin\depot_tools\gcl.py", line 1185, in main
    UploadCL(change_info, argv[3:])
  File "c:\bin\depot_tools\gcl.py", line 783, in UploadCL
    issue, patchset = upload.RealMain(upload_arg, change_info.patch)
  File "c:\bin\depot_tools\upload.py", line 1553, in RealMain
    vcs.UploadBaseFiles(issue, rpc_server, patches, patchset, options, 
files)
  File "c:\bin\depot_tools\upload.py", line 768, in UploadBaseFiles
    UploadFile(filename, file_id, base_content, is_binary, status, True)
  File "c:\bin\depot_tools\upload.py", line 753, in UploadFile
    content_type=ctype)
  File "c:\bin\depot_tools\upload.py", line 376, in Send
    f = self.opener.open(req)
  File "c:\bin\depot_tools\python\lib\urllib2.py", line 364, in open
    response = meth(req, response)
  File "c:\bin\depot_tools\python\lib\urllib2.py", line 471, in 
http_response
    response = self.parent.error(
  File "c:\bin\depot_tools\python\lib\urllib2.py", line 402, in error
    return self._call_chain(*args)
  File "c:\bin\depot_tools\python\lib\urllib2.py", line 337, in _call_chain
    result = func(*args)
  File "c:\bin\depot_tools\python\lib\urllib2.py", line 480, in 
http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error


What version of the product are you using? On what operating system?
vista64

Please provide any additional information below.
There are 1024 files in the change list.  It took about 30 minutes before 
this error occurred