- On my free time, I like reading research papers and comic books, and working on coding projects.
ch3njust1n / smart Goto Github PK
View Code? Open in Web Editor NEWSelf-modifying code at runtime with Large Language Models
License: MIT License
Self-modifying code at runtime with Large Language Models
License: MIT License
Add color to stack_trace so it's clear what part was generated
Dynamic Metaprogramming with Large Language Models
Sections:
Abstract
Introduction
Metaprogramming
-- What is metaprogramming
-- Types of metaprogramming
-- Why use metaprogramming
Code injection
-- What is code injection
-- What are the different types of code injection
-- How to mitigate most common attacks
-- How does this relate to metaprogramming
Related work
Dynamic Metaprogramming with Large Language Models
Results
Considerations
Add a table with
Example from Torch-Struct: Deep Structured Prediction Library paper
Task:
Definition: You can create classes dynamically using the type function.
Example:
def my_method(self):
return self.x * 2
MyClass = type('MyClass', (object,), {'x': 10, 'my_method': my_method})
instance = MyClass()
print(instance.x) # Output: 10
print(instance.my_method()) # Output: 20
Task:
type()
Definition: A metaclass is the class of a class, which Python uses to create and control classes just like classes create and control objects. The default metaclass is type, but you can create your own metaclass by inheriting from type.
Example:
class MyMeta(type):
def __init__(cls, name, bases, dct):
cls.class_created_by = "MyMeta"
super().__init__(name, bases, dct)
class MyClass(metaclass=MyMeta):
pass
print(MyClass.class_created_by) # Output: MyMeta
Task:
Generate a large unit test/ benchmark dataset for paper. Tests should be random and range in complexity.
Example: https://docs.lmql.ai/en/stable/
Translate to Mojo
Definition: This involves changing or extending the behavior of code at runtime, typically by adding, replacing, or modifying methods or attributes in a class or object.
Example:
class MyClass:
pass
def new_method(self):
print("New method")
MyClass.new_method = new_method
instance = MyClass()
instance.new_method() # Prints: "New method"
Task:
new_method
and returns a pointer to the function from exec()
and assigns to MyClass.new_method
Use python's ast package to check the syntax of generated code at runtime.
import ast
def is_code_correct(code):
try:
ast.parse(code)
return True
except SyntaxError:
return False
# test
print(is_code_correct('print("Hello, world!")')) # True
print(is_code_correct('print("Hello, world!')) # False - missing closing quote
Tasks:
pip install mypy
pre-commit-config.yaml
dynamic.yaml
Use GPT-4 to generate code and a faster model with more context like claude-instant-v1-100k
.
User should be able to define their own semantic checking model and the metaprogramming API should optionally expect a semantic checker as input along with the generating model (model
input parameter).
Factory function that given a data structure, it returns a new generative data structure
e.g.
from collections import deque
from generative.structures import convert
q = convert(deque)
Under the hood it could be implemented as follows:
import builtins
from generative import adapt
vars().update({k: adapt.adapt(v) if callable(v) else v for k, v in vars(builtins).items() if not k.startswith("__")})
Import all python modules:
import os
import importlib
import sys
# Get the path of Python's lib directory
lib_path = os.path.dirname(os.__file__)
# Get a list of all .py files in lib directory
module_files = [f for f in os.listdir(lib_path) if f.endswith('.py')]
# Remove the .py extension to get the module names
module_names = [f[:-3] for f in module_files]
# Import the modules
native_modules = []
for name in module_names:
try:
module = importlib.import_module(name)
native_modules.append(module)
except:
pass # If a module fails to import, just skip it
print(native_modules) # This prints the list of imported native modules
Then can modify all native functionality with generative.adapt(<place functionality here>)
, but would need to modify the comprehension so that it recursively applies @adapt
to functions of objects or use metaclasses.generate()
.
Allow user to provide their own callback to decorator so they can specify their own LLM solution
_is_generative
property in decorators and metaclasses indicate a function or class has access to a generative model.
Currently unclear how this could be used, but it gives the rest of the program information on what is generative and what is now.
Here is an example that could be used so that a LLM knows it can also manipulate other parts of the program:
import inspect
import textwrap
from meta import gpt4
all_methods = inspect.getmembers(
cls, predicate=inspect.isfunction
)
available_funcs = [
(
method[0],
textwrap.dedent(inspect.getsource(method[1])),
)
for method in all_methods if method._is_generative
]
func_source = gpt4(f'Generate code with these generative functions: {available_funcs}')
Definition: These are similar to function decorators, but they modify or extend classes instead.
Example:
def my_class_decorator(cls):
cls.decorated = True
return cls
@my_class_decorator
class MyClass:
pass
print(MyClass.decorated) # Output: True
instance = MyClass()
print(instance.decorated) # Output: True
Task:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.