acoustid / mbslave Goto Github PK
View Code? Open in Web Editor NEWMusicBrainz Database Mirror
License: MIT License
mbslave's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
mbslave's Issues
pipx install failing
I tried to install mbslave with pipx as per README, but failed, with pipx log saying:
ERROR: Could not find a version that satisfies the requirement mbslave (from versions: none)
ERROR: No matching distribution found for mbslave
Well …?
mbslave sync fails with missing dbmirror2.pending_data relation
I had this error after upgrading to the latest version:
psycopg2.errors.UndefinedTable: relation "dbmirror2.pending_data" does not exist
cursor.execute('TRUNCATE dbmirror2.pending_data')
I've tried creating a fresh copy and go from there, but it still wouldn't work. Now the whole dbmirror2 schema was missing:
psycopg2.errors.InvalidSchemaName: schema "dbmirror2" does not exist
cursor.execute('TRUNCATE dbmirror2.pending_data')
Seems like the changes introduced by pull-request #5 are causing this error.
Should mbslave/sql/dbmirror2/ReplicationSetup.sql be executed after init to fix this?
I tried that to but ran right into the next problem:
File "/usr/local/lib/python3.12/site-packages/mbslave/replication.py", line 432, in process
cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "artist_release_group_pending_update" does not exist
LINE 1: INSERT INTO artist_release_group_pending_update VALUES (NEW....
^
QUERY: INSERT INTO artist_release_group_pending_update VALUES (NEW.id)
CONTEXT: PL/pgSQL function musicbrainz.a_ins_release_group_mirror() line 3 at SQL statement
This one was a bit surprising as musicbrainz.artist_release_group_pending_update existed. So I'm a bit lost now. May that be a permission issue?
Thanks!
Command mbslave auto-import returned non-zero exit
Each time I deploy and install according to the documentation, I get an error: Command mbslave auto-import returned non-zero exit.what should i do
MusicBrainz schema change release, 2024-05-13
Replication no longer works after new schema is released:
mbslave.replication.MismatchedSchemaError: Mismatched schema sequence, 28 (database) vs 29 (replication packet)
mbslave synce does not ignore replication data for ignored schemas
executing mbslave sync after initializing the DB with ignore=... configured leads to constraint violations when importing data to pending_data:
$ mbslave sync
INFO:mbslave.replication:Downloading https://metabrainz.org/api/musicbrainz/replication-163114-v2.tar.bz2?token=***
INFO:mbslave.replication:Packet was produced at 2023-11-09 00:17:25.761114+00
Traceback (most recent call last):
File "/usr/local/bin/mbslave", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.12/site-packages/mbslave/replication.py", line 795, in main
args.func(config, args)
File "/usr/local/lib/python3.12/site-packages/mbslave/replication.py", line 504, in mbslave_sync_main
process_tar(packet, db, config, ignored_schemas, ignored_tables, schema_seq, replication_seq, hook)
File "/usr/local/lib/python3.12/site-packages/mbslave/replication.py", line 453, in process_tar
importer.load_pending_data(member_file)
File "/usr/local/lib/python3.12/site-packages/mbslave/replication.py", line 349, in load_pending_data
cursor.copy_expert('COPY dbmirror2.pending_data FROM STDIN', fp)
psycopg2.errors.CheckViolation: new row for relation "pending_data" violates check constraint "tablename_exists"
DETAIL: Failing row contains (124502041, statistics.statistic, i, 1511151662, null, {"id" : 17644772, "name" : "count.event.type.5", "value" : 40, "..., null, null).
CONTEXT: COPY pending_data, line 2: "124502041 statistics.statistic i 1511151662 \N {"id" : 17644772, "name" : "count.event.type.5", "val..."
config:
[schemas]
musicbrainz=musicbrainz
statistics=statistics
cover_art_archive=cover_art_archive
event_art_archive=event_art_archive
wikidocs=wikidocs
documentation=documentation
ignore=statistics,wikidocs,documentation
This error might be caused by pull-request #5
Init fails due to violated check constraints
I'm initing the database with these commands:
pipx install 'mbslave'
mbslave init
and running into this error:
...
INFO:mbslave.replication:Loading l_series_work to musicbrainz.l_series_work
INFO:mbslave.replication:Loading l_url_work to musicbrainz.l_url_work
INFO:mbslave.replication:Loading l_work_work to musicbrainz.l_work_work
INFO:mbslave.replication:Loading label to musicbrainz.label
Traceback (most recent call last):
File "/opt/pipx_bin/mbslave", line 8, in <module>
sys.exit(main())
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 787, in main
args.func(config, args)
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 297, in mbslave_auto_import_main
load_tar(url, fileobj, db, config, config.schemas.ignored_schemas, config.tables.ignored_tables)
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 258, in load_tar
cursor.copy_expert('COPY {} FROM STDIN'.format(fulltable), tar.extractfile(member))
psycopg2.errors.CheckViolation: new row for relation "label" violates check constraint "label_label_code_check"
DETAIL: Failing row contains (294731, dacec7dc-806e-4f1a-ab41-cc0c46b297e0, beau by Republic, 2022, 10, 9, null, null, null, 202210, 1, 7741, , 0, 2024-05-14 21:06:04.842073+00, f).
CONTEXT: COPY label, line 39973: "294731 dacec7dc-806e-4f1a-ab41-cc0c46b297e0 beau by Republic 2022 10 9 \N \N \N 202210 1 7741 0 202..."
Traceback (most recent call last):
File "/opt/pipx_bin/mbslave", line 8, in <module>
sys.exit(main())
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 787, in main
args.func(config, args)
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 640, in mbslave_init_main
run_script('mbslave auto-import')
File "/opt/pipx/venvs/mbslave/lib/python3.10/site-packages/mbslave/replication.py", line 593, in run_script
subprocess.run(script, check=True, shell=True)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'mbslave auto-import' returned non-zero exit status 1.
Database creation no longer works after a new schema is released
Commands: pip install 'mbdata[replication]' --break-system-packages, mbslave init --create-database
Error log:
...
2024-05-26 20:47:17 INFO:mbdata.replication:Loading link_type to musicbrainz.link_type
2024-05-26 20:47:17 Traceback (most recent call last):
2024-05-26 20:47:17 File "/usr/local/bin/mbslave", line 8, in <module>
2024-05-26 20:47:17 sys.exit(main())
2024-05-26 20:47:17 ^^^^^^
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 803, in main
2024-05-26 20:47:17 args.func(config, args)
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 297, in mbslave_auto_import_main
2024-05-26 20:47:17 load_tar(url, fileobj, db, config, config.schemas.ignored_schemas, config.tables.ignored_tables)
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 258, in load_tar
2024-05-26 20:47:17 cursor.copy_expert('COPY {} FROM STDIN'.format(fulltable), tar.extractfile(member))
2024-05-26 20:47:17 psycopg2.errors.InvalidTextRepresentation: invalid input syntax for type integer: "2014-05-18 09:46:23.72719+00"
2024-05-26 20:47:17 CONTEXT: COPY link_type, line 1, column priority: "2014-05-18 09:46:23.72719+00"
2024-05-26 20:47:17
2024-05-26 20:47:17 Traceback (most recent call last):
2024-05-26 20:47:17 File "/usr/local/bin/mbslave", line 8, in <module>
2024-05-26 20:47:17 sys.exit(main())
2024-05-26 20:47:17 ^^^^^^
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 803, in main
2024-05-26 20:47:17 args.func(config, args)
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 656, in mbslave_init_main
2024-05-26 20:47:17 run_script('mbslave auto-import')
2024-05-26 20:47:17 File "/usr/local/lib/python3.11/dist-packages/mbdata/replication.py", line 609, in run_script
2024-05-26 20:47:17 subprocess.run(['bash', '-euxc', script], check=True)
2024-05-26 20:47:17 File "/usr/lib/python3.11/subprocess.py", line 571, in run
2024-05-26 20:47:17 raise CalledProcessError(retcode, process.args,
2024-05-26 20:47:17 subprocess.CalledProcessError: Command '['bash', '-euxc', 'mbslave auto-import']' returned non-zero exit status 1.
It seems like the PyPI package has not been updated yet.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.