Comments (5)
Supabase only supports Trusted Language Extensions(TLE) supabase/supabase#14600 (comment) for security of custom extensions.
Many extensions provide functions whose implementation is written in C, and creating them in a database means that the compiled C code is “dynamically linked” into your running Postgres process. These dynamically-loaded libraries can now access every aspect of your running database process, right down to raw memory. They are essentially database superusers on steroids. Because of this, C is an “untrusted language” and installing extensions written in C requires filesystem access.
There is a Rust tle implementation: https://github.com/tcdi/plrust
However, pgvecto.rs is hard to converted to TLE as it uses ipc/mmap, which is absolutely forbidden.
from pgvecto.rs.
Unfortunately there's no way to install custom extension on postgres supabase/supabase#14235. I would suggest you tried pgvecto.rs through docker first with your requirement. Such as
docker run --name pgvecto-rs-demo -e POSTGRES_PASSWORD=mysecretpassword -p 5432:5432 -d tensorchord/pgvecto-rs:latest
I believe we can easily support this for the drizzle-orm. The only syntax that differs from pgvector is the index creation command. All other query commands are exactly the same as pgvector. Let me give it a try, and I will submit a pull request for drizzle soon.
from pgvecto.rs.
I would also like to hear your scenarios (what kind of filter condition you'd like to use). pgvecto.rs has made significant efforts to support various filter mode (prefilter/postfilter/brute force, and we are working on bitmap pushdown to use postgres index on other columns) and optimize performance. If it helps, I can provide further guidance on performance optimization. Thank you!
from pgvecto.rs.
I would need cross filtering for my usecases. Mostly my usecases are filtering by normal sql columns with vector similarity as well as one of the columns in the where clause.
pre/Post filtering has a higher probably of no/spare results or irrelevant results.
For example searching for documents with a tag and with similarity (vector column) assigned to a particular team.
from pgvecto.rs.
We are also building an example (https://github.com/kemingy/ragen/blob/main/ragen/client.py#L77-L82) similar to your scenario, using vector search with tag filter. And it worked well based on our example.
For pgvecto.rs, The default prefilter will ensure that the vector index returns a number of results equal to vectors.k
and meets the specified filter condition.
May I ask what your typical filter condition selection rate is (what percentage of data satisfy your filter condition)? Also what is the "cross filtering" method you mentioned??
from pgvecto.rs.
Related Issues (20)
- feat(ecosystem): Telemetry HOT 1
- Container image ignores additional shared_preload_libraries added via postgresql.conf HOT 6
- ERROR: pgvecto.rs: IPC connection is closed unexpected. v0.2.0 HOT 4
- feat: allow multiple subscripts HOT 5
- chore: Fix permission for pg_vector_index_stat
- CREATE INDEX gives: ERROR: pgvecto.rs: The index is already existing in the background worker. HOT 5
- ci: build aarch64 release with given aarch64 bindings
- pgvecto.rs 0.2.1 much slower than pgvector 0.6.1? HOT 9
- test: Test pgvecto.rs FDW against postgres with pgvector HOT 3
- chore: upgrade script for 0.3.0 HOT 3
- Unexpected behavior with NULL HOT 1
- epic
- epic: Release 0.3.0 HOT 1
- benchmark: INT8 benchmark HOT 1
- test: 0.3 release test HOT 1
- chore: sync return when index creation
- Need a docker compose example HOT 4
- epic: Support SDKs in different programming languages HOT 4
- bug(cnpg): type "vector" does not exist HOT 1
- The default max_stack_depth parameter in the container image may be too small HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pgvecto.rs.