Comments (11)
Okay, yeah this should totally work. They introduced the ability to define PKs, foreign keys, and indexes on partitioned tables in PG 11. One small note about the above code sample. There are actually helper methods in partitioned model classes that will let you create partitions without directly using ActiveRecord::Base.connection
. The helpers imply the partition key and figures out what underlying method to call (create_list_partition_of
vs create_range_partition_of
). This should probably be documented better.
SomeListRecords.create_partition(name: "some_unique_table_name", values: [some_foreign.id])
from pg_party.
That's a great question and I'll have to get back to you. Something like that might be supported in PG 12+
from pg_party.
I tried this setup and it seems to work (I am running postgres 12.3 though), but I'm not sure if there are hidden problems that can occur in regular usage.
Given the above setup, this spec passes:
describe SomeListRecord do
describe 'partitions by some_foreign' do
let!(:some_foreign_1) { create :some_foreign }
let!(:some_foreign_2) { create :some_foreign }
before do
[some_foreign_1, some_foreign_2].each do |some_foreign|
ActiveRecord::Base
.connection
.create_list_partition_of(
:some_list_records, name: some_foreign.name, partition_key: :some_foreign_id, values: [some_foreign.id]
)
end
end
let!(:some_list_record_1) { create :some_list_record, some_foreign: some_foreign_1 }
let!(:some_list_record_2) { create :some_list_record, some_foreign: some_foreign_1 }
let!(:some_list_record_3) { create :some_list_record, some_foreign: some_foreign_2 }
it 'returns some_foreign_1 records' do
expect(SomeListRecord.in_partition(some_foreign_2.name).all).to contain_exactly(some_list_record_1, some_list_record_2)
end
it 'returns some_foreign_2 record' do
expect(SomeListRecord.in_partition(some_foreign_2.name).all).to contain_exactly(some_list_record_3)
end
it 'has 2 corresponding partitions' do
expect(SomeListRecord.partitions).to contain_exactly(some_foreign_1.name, some_foreign_2.name)
end
end
end
from pg_party.
awesome, thanks for the response.
from pg_party.
just fyi, SomeListRecords.create_partition(name: some_foreign.name, values: [some_foreign.id])
instead of
ActiveRecord::Base
.connection
.create_list_partition_of(
:some_list_records, name: some_foreign.name, partition_key: :some_foreign_id, values: [some_foreign.id]
)
fails the test above.
Factorybot creates instances with id: nil
, even though .errors
is empty and .persisted?
/.valid?
are true.
from pg_party.
Huh, that's interesting. What does your SomeListRecords
class look like?
from pg_party.
these are the models:
class SomeListRecords < ActiveRecord::Base
list_partition_by :some_foreign_id
belongs_to :some_foreign
belongs_to :another_foreign
end
class SomeForeign < ActiveRecord::Base
has_many :messages
has_many :some_list_records
end
these are the migrations:
create_list_partition :some_list_records, partition_key: :some_foreign_id do |t|
t.string :uuid, null: false
t.references :some_foreign, foreign_key: true, index: false
t.references :another_foreign, foreign_key: true
t.timestamps null: false
end
add_index :some_list_records_template, [:uuid, :some_foreign_id]
add_index :some_list_records_template, [:some_foreign_id, :created_at]
create_table :some_foreigns do |t|
t.string :name, null: false, unique: true
t.timestamps null: false
end
and these are my factories:
FactoryBot.define do
factory :some_list_record, class: SomeListRecord do
sequence(:uuid) { |n| 'uuid_' + n.to_s }
association :some_foreign, factory: :some_foreign
association :another_foreign, factory: :another_foreign
end
end
FactoryBot.define do
factory :some_foreign, class: SomeForeign do
sequence(:name) { |n| 'some_foreign' + n.to_s }
end
end
anything pop out?
from pg_party.
created a sample app that demonstrates this:
https://github.com/mrlevitas/partition-example
hope it helps debugging.
from pg_party.
Thanks for putting this together. I have a pretty busy week but can hopefully look into it soon
from pg_party.
Sorry it took me so long to look into this. I think this is actually an rspec
issue. By default, it starts a transaction for each spec and rolls it back afterwards. This is generally fine in most cases where you're just performing inserts, updates, deletes. But when you introduce DDL operations things start to break down.
So by running ActiveRecord::Base.connection.create_list_partition_of
, I think that was being executed in a separate connection outside of the transaction.
If you'd like to perform DDL in your tests you might want to consider using database_cleaner
with the truncation
strategy: https://github.com/rkrage/pg_party/blob/master/spec/spec_helper.rb#L59-L69
It's also possible that truncation is natively supported in rspec
as well and I'm just out of the loop.
from pg_party.
Gonna close this for now, but feel free to open it again if you're still experiencing issues!
from pg_party.
Related Issues (20)
- Schema file contains partion tables HOT 8
- Multiple Key Partition HOT 3
- Update readme for 1.2.0 release
- Question: schema_exclude_partitions = false development workflow HOT 7
- Feature request: select range partitions by one of date HOT 3
- Fix ruby 2.7 keyword argument warnings HOT 2
- Use #create_list_partition_of in rake task? HOT 1
- rspec tests fail unless using structure.sql HOT 2
- Explanation needed: Are many "partition tables" a problem? HOT 2
- Convert a normal table to partitioned table HOT 2
- ActiveRecord doesn't support composite primary keys? HOT 4
- Rails 6.1 support HOT 2
- Address Rails 6.1 deprecation warnings HOT 2
- Any talks to move some of these features into Rails core? HOT 7
- No changelog entry for 1.5.0 HOT 3
- The way I setup on Rails 6.1 HOT 1
- Migration from `partitioned` gem to `pg_party` HOT 1
- pg_dump: error: cross-database references are not implemented HOT 1
- create_range_partition and Ruby 3 HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pg_party.