How to handle database restarts about alchy

[dgilland]

So this is not really an issue with alchy but more of an issue to be addressed with SQLAlchemy and whatever web framework you are using.

There are a few approaches you can take for this:

• Ensure that your error handling rolls back any database transactions and properly destroys the db session.
• Ensure that your database connection is operational when it is checked out from the connection pool.
• Others?

The link below describes how someone implemented a pool listener to "test" the database connection before it was checked out from the pool and how someone implemented a proxy connection to handle operation errors:

http://stackoverflow.com/questions/3033234/handle-mysql-restart-in-sqlalchemy

Something similar could be done for PostgreSQL.

However, that does come with some overhead so might not be worth it on your end. But you'll definitely want to take a look at your error handling and see if you can take care of the db session/transaction when an error occurs.

None of this is specific to alchy though.

UPDATE: That StackOverflow link is a little old. SQLAlchemy now has event.listens_for which can be used for similar results: http://docs.sqlalchemy.org/en/latest/core/event.html

[mtrofimm]

Thanks for information.
You are right, it's not a issue with alchy.

The whole test case is the following:

1. I run a query using model: (do I need to rollback/close session here??)

query = self.vehicle_files.with_entities(VehicleFilePack)
if provider_name:
   query=query.filter(VehicleFilePack.data_source['provider'].cast(String) == provider_name)
if provider_id:
    query=query.filter(VehicleFilePack.data_source['uniq_record'].cast(String) == provider_id)
file_packs = query.params(vin=vin, access_level=access_level).all()
# some additoinal filtering  of file_packs here
query.session.expunge_all()
return file_packs

1. Then I restart dabase server.
2. Run the same query again and got 500 with message

    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) terminating connection due to administrator command
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

1. the next execution of the query returns:

cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) terminating connection due to administrator command
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

I noticed that:

1. there implicit transaction start before first select

2015-12-16 12:48:13,905 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
2015-12-16 12:48:13,905 - sqlalchemy.engine.base.Engine - INFO - BEGIN (implicit)

but no commit at the end :(
2. checkout event is fired only once before the first execution of the query.
3. There is no checkin event in the first execution.
4. In the second execution (after db restart) there are 'invalidate' and 'checkin' (this with connection=None) events.

[mtrofimm]

I changed the code and added:
query.session.commit()
after expunge_all()

In such case there is a a 'checkin' event in the first execution and checkout event in the second one.
Now it's possible to handle OperationalError in event listener.

[dgilland]

In your example, you are only reading data so there's nothing to commit. So no need to add one.

Usually after your web request, you'd want to call session.remove() to release any existing transactional/connection resources still being held and return the connection back to the pool.

[mtrofimm]

there is not method remove in session class :(

[dgilland]

http://docs.sqlalchemy.org/en/latest/orm/contextual.html#sqlalchemy.orm.scoping.scoped_session.remove