Comments (11)
Fixed in Release 1.5.2
from grate.
@koenekelschot sent repro file via email.
from grate.
I am seeing the same error while trying to run an Up
script created using pg_dump
that includes dollar-quoted strings.
Interestingly enough I get another error when trying to use a script created by pg_dump
with the --disable-dollar-quoting
option.
Severity: ERROR
SqlState: 42601
MessageText: unterminated quoted string at or near "'"
Position: 1342
File: scan.l
Line: 1145
from grate.
Reverting to grate 1.5.0 makes it so the dollar-quoted dump restores successfully.
from grate.
@Westat-Transportation is there any chance you can share the script with dollar quoting disabled? I could use that as a test case for my proposed fix.
from grate.
Sure, can I send it privately? It's for a LOB system db and its schema is proprietary.
from grate.
I'm not sure if you can see the email address in my profile. But I don't need the whole script, I suspect there is a query using multiple single quotes (as an escape). As in this contrived test case: https://github.com/erikbra/grate/pull/358/files#diff-1527405d352f46baa59fd9735ebf1a6996447c0b3ccf3badad7bdb5d5c49615a (line 41).
That's the part I hope to verify.
from grate.
Folks, gave grate 1.5.2 a go today and hit similar crashes on $$ terminated functions. The error message mentioned a line where $$ was declared and immediately followed by another keyword:
Error running script "001-db_schema.sql": 42601: unterminated dollar-quoted string at or near "$$TRUNCATE TABLE gis.tps_tbw_gps_places"
So I edited the .sql file and made sure all of the AS $$
declarations were followed by a line break with the next statement on the next line. When I did that I got another error:
Error running script "001-db_schema.sql": 42601: unterminated quoted string at or near "'"
Went back to 1.5.0 and it worked with both versions of the file 001-db_schema.sql
. So I think there may still be room for improvement in parsing files using $$ termination. I will email the git repo which is running these separately -- that will allow you to see my edits from this morning.
from grate.
I am doing some local debugging against my files here. Will update this later if I can make it work.
from grate.
No luck, my attempts at tweaking the RegEx expressions did not work. I modified the code to write out the parsed statements so I can see where it was on the file when it bombed and found out that the following conditions were problematic for it:
- semicolons inside single quotes as part of text concatenation expressions (
col_name || '; '::text)
. - DO code blocks delimited by
$do$
from grate.
@erikbra and @koenekelschot, I decided to take my modified code which records to a separate file the actual split statements that were sent to the db (just now realized I could have change verbosity, oh well) and bring over what the RegExes were in 1.5.0 and realized that grate was not really splitting PostgreSQL statements back in 1.5.0. The whole contents of a database dump were being sent to the server via npgsql.
Given the difficulties we are having to get this splitting to work in 1.5.1 and 1.5.2 what about introducing a CLI switch which basically turns that off on grate? I realize this mode would not work for cases like bugs/334-pgsql-concurrent. But it would allow folks like me to keep using the latest until a better approach for statement splitting can be developed.
Just a thought.
from grate.
Related Issues (20)
- Error with Grate 1.7.0 - Read-only file system HOT 3
- Grate internal scripts fail when using CreateDatabase scripts HOT 4
- Upgrade failing for MySQL with syntax error using v1.7.0 HOT 8
- Suppressing logging of internal grate migrations doesn't appear to be working correctly HOT 3
- Scripts in internal Grate migration tables are always registered as "run" HOT 3
- --drop without continuing migration use case HOT 1
- Create database fails when an empty createDatabase folder is present
- Should CONTRIBUTING.md be updated to list .NET 8 HOT 2
- runAfterCreateDatabase are run even if database isn't created from scratch by grate HOT 3
- GrateScritpsRun everytime i run the tool HOT 2
- Question - How to mark one-time migration as run HOT 3
- Doesn't work on console app NET 8 using grate.sqlserver HOT 1
- Running Grate on RoundhousE database - all scripts are new HOT 8
- Baseline creates orphan version_id in ScriptsRun HOT 1
- This is still not fixed. Running grate on a postgres database which had previously been migrated using roundhouse does not work. HOT 2
- Publish linux/arm64 images HOT 2
- v1.8.0 MariaDB/MySQL fix casing scripts seem to break with MySQL
- Migration fails to complete with transaction error HOT 1
- [FEATURE] : Auto Create ALTER TABLE scripts when an existing table adds column to existing SQL Script. HOT 2
- Issue while creating ONLINE indexes for localhost. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grate.