chromium / codesearch-py

Python library for accessing Chromium CodeSearch via https://cs.chromium.org

Chromium

Chromium is an open-source browser project that aims to build a safer, faster, and more stable way for all users to experience the web.

The project's web site is https://www.chromium.org.

To check out the source code locally, don't use git clone! Instead, follow the instructions on how to get the code.

Documentation in the source is rooted in docs/README.md.

Learn how to Get Around the Chromium Source Code Directory Structure.

For historical reasons, there are some small top level directories. Now the guidance is that new top level directories are for product (e.g. Chrome, Android WebView, Ash). Even if these products have multiple executables, the code should be in subdirectories of the product.

If you found a bug, please file it at https://crbug.com/new.

codesearch-py's Issues

Server requests return HTML due to redirect to new codesearch

API requests currently return HTML instead of JSON, resulting in an error. @asankah advised that this is due to requests to cs.chromium.org being redirected to the new codesearch at source.chromium.org, which now requires an API key.

For example, this test script:

g_cs = codesearch.CodeSearch(should_cache=True, source_root='.')
response = g_cs.SendRequestToServer(codesearch.CompoundRequest(
    search_request=[
    codesearch.SearchRequest(query='hello world',
                             return_line_matches=True,
                             lines_context=0,
                             max_num_results=10)
    ]))

Currently hits this error:

Traceback (most recent call last):
  File "C:\Users\jessemckenna\AppData\Roaming\Sublime Text 3\Packages\ChromiumXRefs\third_party\codesearch\messages.py", line 204, in FromJsonString
    d = json.loads(s)
  File "C:\src\depot_tools\bootstrap-3_8_0_chromium_8_bin\python3\bin\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "C:\src\depot_tools\bootstrap-3_8_0_chromium_8_bin\python3\bin\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\src\depot_tools\bootstrap-3_8_0_chromium_8_bin\python3\bin\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Printing the server response, result, from SendRequestToServer before it is passed to FromJSONString shows an HTML response from source.chromium.org.

How the codesearch site built ?

the new ui experience is great, how to build the code search site ? 😄 , it is amazing for large project mangement.

[proposal] Register codesearch-py PyPI name

It would be interesting to submit your package to PyPI.

http://peterdowns.com/posts/first-time-with-pypi.html

Deal with platform specific signatures

The backend can return platform specific signatures in response to xref_search. These signatures don't show up in an annotation_request for the same file. This breaks GetDisplayName().

AssertionError in test_file_cache.py

Running python -m unittest discover shows one test error.

======================================================================
FAIL: test_with_cache_dir (codesearch.test_file_cache.TestFileCache)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/ports/devel/py-codesearch/work/codesearch-py-7ed08c1/codesearch/test_file_cache.py", line 37, in test_with_cache_dir
    self.assertEqual('hello'.encode('utf-8'), g.get('foo'))
AssertionError: 'hello' != ''

----------------------------------------------------------------------
Ran 32 tests in 1.248s

FAILED (failures=1)
*** Error code 1

Duplicate results in Xref Search

Due to platform specific signatures, there could be multiple matches in a xref search. Need a way to squash those into a single xrefnode.

GetAnnotations options

When I call csfile.GetAnnotations(), it's missing some important annoations types, namely AnnotationTypeValue.LINK_TO_DEFINITION.

I can get them by calling GetAnnotationsForFile but that bypasses the CsFile cache. Can you add AnnotationTypeValue.LINK_TO_DEFINITION to the default types?

Following links to `out/win-Debug/gen` etc. paths don't work

The search index stores platform specific generated files in the following directories (as of this writing):

• out/Debug: for Linux
• out/win-Debug: for Windows
• out/android-Debug: for Android
• out/chromeos-Debug: for Chrome OS.

When references to these files show up, the library should "intelligently" map the path to whatever output directories exist on the local host.

CodeSearch response may not be valid UTF-8

The server response may include snippets that are not valid UTF-8 while Python's json.loads() assumes that the content is valid UTF-8.

Signature lookups fail when the source file being examined is locally modified

This doesn't happen all the time and is only an issue when the optimistic symbol mapping code fails. We need something much more robust than what we have now.