blarapp / code-base-agent Goto Github PK
View Code? Open in Web Editor NEWCode agents for LLMs
Home Page: https://blar.io
License: MIT License
Code agents for LLMs
Home Page: https://blar.io
License: MIT License
It would be interesting to see if/how blar
performs against the SWE-Bench benchmarks:
[ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues?
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan
Language models have outpaced our ability to evaluate them effectively, but for their future development it is essential to study the frontier of their capabilities. We consider real-world software engineering to be a rich, sustainable, and challenging testbed for evaluating the next generation of language models. We therefore introduce SWE-bench, an evaluation framework including 2,294 software engineering problems drawn from real GitHub issues and corresponding pull requests across 12 popular Python repositories. Given a codebase along with a description of an issue to be resolved, a language model is tasked with editing the codebase to address the issue. Resolving issues in SWE-bench frequently requires understanding and coordinating changes across multiple functions, classes, and even files simultaneously, calling for models to interact with execution environments, process extremely long contexts and perform complex reasoning that goes far beyond traditional code generation. Our evaluations show that both state-of-the-art proprietary models and our fine-tuned model SWE-Llama can resolve only the simplest issues. Claude 2 and GPT-4 solve a mere 4.8% and 1.7% of instances respectively, even when provided with an oracle retriever. Advances on SWE-bench represent steps towards LMs that are more practical, intelligent, and autonomous.
Describe the bug
The code fails when trying to create the graph of the pvlib repo
Trace
Traceback (most recent call last):
File "//build_graph.py", line 16, in <module>
build_graph(repo_name)
File "//build_graph.py", line 12, in build_graph
graph_constructor.build_graph(repo_name)
File "/root/miniconda3/lib/python3.9/site-packages/blar_graph/graph_construction/core/graph_builder.py", line 264, in build_graph
relationships.extend(self._relate_imports(imports))
File "/root/miniconda3/lib/python3.9/site-packages/blar_graph/graph_construction/core/graph_builder.py", line 148, in _relate_imports
targetId = self.global_imports.get(import_alias)
TypeError: unhashable type: 'list'
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Get the correct graph
Screenshots
Desktop (please complete the following information):
Describe the bug
When trying to create a graph of the code-base-agent repo by running the run.py with graph_constructor.build_graph("src") it throws the following error when processing
code-base-agent/src/blar_graph/db_managers/init.py
Traceback (most recent call last):
File "/home/jose/Desktop/Trabajo/blar/code-base-agent/src/blar_graph/graph_construction/core/graph_builder.py", line 59, in _scan_directory
processed_nodes, relations, file_imports = self.parser.parse_file(
File "/home/jose/Desktop/Trabajo/blar/code-base-agent/src/blar_graph/graph_construction/languages/python/python_parser.py", line 76, in parse_file
return [], [], self.parse_init(file_path, root_path)
File "/home/jose/Desktop/Trabajo/blar/code-base-agent/src/blar_graph/graph_construction/languages/python/python_parser.py", line 97, in parse_init
new_import_path = import_path + "." + import_statement.text.decode()
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'
To Reproduce
Steps to reproduce the behavior:
Not really a feature request per se, but came across this project after it was mentioned in another issue (Ref), and figured I would create an issue to share the same info here in case it's useful.
I notice that you're already using tree-sitter
, so it may not be a lot of extra effort to make use of tree-sitter-graph
and/or the stack-graphs
project (eg. tree-sitter-stack-graphs-javascript
, etc)
From watching the demo video on linkedin + briefly skimming this repo, it looks like there might be some useful crossovers, and it may allow you to add support for a whole bunch more languages without needing to re-invent the wheel to do so.
A few notes/links/references I recently collated RE: stack graphs + related libs:
Stack Graphs (an evolution of Scope Graphs) sound like they could be really interesting/useful with regards to code navigation, symbol mapping, etc. Perhaps we could use them for module identification, or variable/function identifier naming stabilisation or similar?
- https://github.blog/changelog/2024-03-14-precise-code-navigation-for-typescript-projects/
Precise code navigation is now available for all TypeScript repositories.
Precise code navigation gives more accurate results by only considering the set of classes, functions, and imported definitions that are visible at a given point in your code.Precise code navigation is powered by the stack graphs framework.
You can read about how we use stack graphs for code navigation and visit the stack graphs definition for TypeScript to learn more.
- https://github.blog/2021-12-09-introducing-stack-graphs/
Introducing stack graphs
Precise code navigation is powered by stack graphs, a new open source framework we’ve created that lets you define the name binding rules for a programming language using a declarative, domain-specific language (DSL). With stack graphs, we can generate code navigation data for a repository without requiring any configuration from the repository owner, and without tapping into a build process or other CI job.
- LOTS of interesting stuff in this post..
As part of developing stack graphs, we’ve added a new graph construction language to Tree-sitter, which lets you construct arbitrary graph structures (including but not limited to stack graphs) from parsed CSTs. You use stanzas to define the gadget of graph nodes and edges that should be created for each occurrence of a Tree-sitter query, and how the newly created nodes and edges should connect to graph content that you’ve already created elsewhere.
- https://github.com/tree-sitter/tree-sitter-graph
tree-sitter-graph
The tree-sitter-graph library defines a DSL for constructing arbitrary graph structures from source code that has been parsed using tree-sitter.- https://marketplace.visualstudio.com/items?itemName=tree-sitter.tree-sitter-graph
tree-sitter-graph
support for VS Code
This language extension for VS Code provides syntax support fortree-sitter-graph
files.Why aren’t we using the Language Server Protocol (LSP) or Language Server Index Format (LSIF)?
To dig even deeper and learn more, I encourage you to check out my Strange Loop talk and the
stack-graphs
crate: our open source Rust implementation of these ideas.
- https://github.com/github/stack-graphs
Stack graphs
The crates in this repository provide a Rust implementation of stack graphs, which allow you to define the name resolution rules for an arbitrary programming language in a way that is efficient, incremental, and does not need to tap into existing build or program analysis tools.- https://docs.rs/stack-graphs/latest/stack_graphs/
- https://github.com/github/stack-graphs/tree/main/languages
This directory contains stack graphs definitions for specific languages.
- https://github.com/github/stack-graphs/tree/main/languages/tree-sitter-stack-graphs-javascript
tree-sitter-stack-graphs
definition for JavaScript
This project definestree-sitter-stack-graphs
rules for JavaScript using thetree-sitter-javascript
grammar.The command-line program for
tree-sitter-stack-graphs-javascript
lets you do stack graph based analysis and lookup from the command line.
cargo install --features cli tree-sitter-stack-graphs-javascript
tree-sitter-stack-graphs-javascript index SOURCE_DIR
tree-sitter-stack-graphs-javascript status SOURCE_DIR
tree-sitter-stack-graphs-javascript query definition SOURCE_PATH:LINE:COLUMN
- https://github.com/github/stack-graphs/tree/main/languages/tree-sitter-stack-graphs-typescript
tree-sitter-stack-graphs
definition for TypeScript
This project definestree-sitter-stack-graphs
rules for TypeScript using thetree-sitter-typescript
grammar.The command-line program for
tree-sitter-stack-graphs-typescript
lets you do stack graph based analysis and lookup from the command line.- https://dcreager.net/talks/2021-strange-loop/
- Redirects to https://dcreager.net/talks/stack-graphs/
Incremental, zero-config Code Navigation using stack graphs.
In this talk I’ll describe stack graphs, which use a graphical notation to define the name binding rules for a programming language. They work equally well for dynamic languages like Python and JavaScript, and for static languages like Go and Java. Our solution is fast — processing most commits within seconds of us receiving your push. It does not require setting up a CI job, or tapping into a project-specific build process. And it is open-source, building on the tree-sitter project’s existing ecosystem of language tools.
- https://www.youtube.com/watch?v=l2R1PTGcwrE
"Incremental, zero-config Code Nav using stack graphs" by Douglas Creager
- https://media.dcreager.net/dcreager-strange-loop-2021-slides.pdf
- https://media.dcreager.net/dcreager-2022-ucsc-lsd-slides.pdf
- https://docs.github.com/en/repositories/working-with-files/using-files/navigating-code-on-github
GitHub has developed two code navigation approaches based on the open source tree-sitter and stack-graphs library:
- Search-based - searches all definitions and references across a repository to find entities with a given name
- Precise - resolves definitions and references based on the set of classes, functions, and imported definitions at a given point in your code
To learn more about these approaches, see "Precise and search-based navigation."
- https://docs.github.com/en/repositories/working-with-files/using-files/navigating-code-on-github#precise-and-search-based-navigation
Precise and search-based navigation
Certain languages supported by GitHub have access to precise code navigation, which uses an algorithm (based on the open source stack-graphs library) that resolves definitions and references based on the set of classes, functions, and imported definitions that are visible at any given point in your code. Other languages use search-based code navigation, which searches all definitions and references across a repository to find entities with a given name. Both strategies are effective at finding results and both make sure to avoid inappropriate results such as comments, but precise code navigation can give more accurate results, especially when a repository contains multiple methods or functions with the same name.- https://pl.ewi.tudelft.nl/research/projects/scope-graphs/
Scope Graphs | A Theory of Name Resolution
Scope graphs provide a new approach to defining the name binding rules of programming languages. A scope graph represents the name binding facts of a program using the basic concepts of declarations and reference associated with scopes that are connected by edges. Name resolution is defined by searching for paths from references to declarations in a scope graph. Scope graph diagrams provide an illuminating visual notation for explaining the bindings in programs.
Originally posted by @0xdevalias in 0xdevalias/chatgpt-source-watch#11
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.