facebook / squangle

SQuangLe is a C++ API for accessing MySQL servers

SQuangLe

Overview

SQuangLe is a C++ MySQL client library built on top of WebScaleSQL's C client library. It does not require a WebScaleSQL server - WebScaleSQL, MySQL, MariaDB, and Percona Server should all work fine.

Current Status

SQuangLe is not supported as a standalone project; it is released as a dependency of HHVM's async mysql extension to the Hack language.

Features

• Object-oriented API
• Query builder with automatic escaping
• Asynchronous query execution

License

SQuangLe is BSD-licensed. We also provide an additional patent grant. Please see the LICENSE file.

Contributing

Please see CONTRIBUTING.md

squangle's Issues

AsyncConnectionPool::CleanUpTimer segfaults

We (slack) have been seeing a very slow trickle of segfaults from code paths in the cleanup timer in our production environment. This issue is not new, it's been occurring for a while. I do not yet have a repro for these segfaults.

Relevant version of squangle we are running:

┌─(cy@zebu:~/sl/hhvm/third-party/squangle/src)
└─(29)% git log -n 1 --pretty=short  
commit 9b3d6adf34d4f1ec1c1713a54b9def947384b17b (HEAD)
Author: Jay Edgar <[email protected]>

    Update state_ inside mutex

There are two unique stack traces we see. The first is more frequent and appears to occur on a call to std::unordered_map::erase():

0:string"raise at ../sysdeps/unix/sysv/linux/raise.c:51"
1:string"HPHP::bt_handler at /build/hhvm/hphp/runtime/base/crash-reporter.cpp:270"
2:string"std::_Hashtable<facebook::common::mysql_client::PoolKey, at /usr/include/c++/7/bits/hashtable.h:1627"
3:string"std::_Hashtable<facebook::common::mysql_client::PoolKey, at /usr/include/c++/7/bits/hashtable.h:1864"
4:string"std::_Hashtable<facebook::common::mysql_client::PoolKey, at /usr/include/c++/7/bits/hashtable.h:755"
5:string"std::unordered_map<facebook::common::mysql_client::PoolKey, at /usr/include/c++/7/bits/unordered_map.h:797"
6:string"facebook::common::mysql_client::AsyncConnectionPool::ConnStorage::cleanupConnections at /build/hhvm/third-party/squangle/squangle/mysql_client/AsyncConnectionPool.cpp:619"
7:string"facebook::common::mysql_client::AsyncConnectionPool::CleanUpTimer::timeoutExpired at /build/hhvm/third-party/squangle/squangle/mysql_client/AsyncConnectionPool.cpp:519"

The second one looks like use of an invalid map iterator ('ref_iter->second' where ref_iter is probably pointing to end()?) :

0:string"raise at ../sysdeps/unix/sysv/linux/raise.c:51"
1:string"HPHP::bt_handler at /build/hhvm/hphp/runtime/base/crash-reporter.cpp:270"
2:string"facebook::common::mysql_client::AsyncMysqlClient::activeConnectionRemoved at /build/hhvm/third-party/squangle/src/squangle/mysql_client/AsyncMysqlClient.h:357"
3:string"facebook::common::mysql_client::MysqlConnectionHolder::~MysqlConnectionHolder at /build/hhvm/third-party/squangle/squangle/mysql_client/Connection.cpp:63"
4:string"facebook::common::mysql_client::MysqlConnectionHolder::~MysqlConnectionHolder at /build/hhvm/third-party/squangle/squangle/mysql_client/Connection.cpp:64"
5:string"std::default_delete<facebook::common::mysql_client::MysqlPooledHolder>::operator() at /usr/include/c++/7/bits/unique_ptr.h:78"
6:string"std::unique_ptr<facebook::common::mysql_client::MysqlPooledHolder, at /usr/include/c++/7/bits/unique_ptr.h:263"
7:string"__gnu_cxx::new_allocator<std::_List_node<std::unique_ptr<facebook::common::mysql_client::MysqlPooledHolder, at /usr/include/c++/7/ext/new_allocator.h:140"
8:string"std::allocator_traits<std::allocator<std::_List_node<std::unique_ptr<facebook::common::mysql_client::MysqlPooledHolder, at /usr/include/c++/7/bits/alloc_traits.h:487"
9:string"std::__cxx11::list<std::unique_ptr<facebook::common::mysql_client::MysqlPooledHolder, at /usr/include/c++/7/bits/stl_list.h:1815"
10:string"facebook::common::mysql_client::AsyncConnectionPool::ConnStorage::cleanupConnections at /build/hhvm/third-party/squangle/squangle/mysql_client/AsyncConnectionPool.cpp:613"
11:string"facebook::common::mysql_client::AsyncConnectionPool::CleanUpTimer::timeoutExpired at /build/hhvm/third-party/squangle/squangle/mysql_client/AsyncConnectionPool.cpp:519"
12:string"folly::AsyncTimeout::libeventCallback at /build/hhvm/third-party/folly/src/folly/io/async/AsyncTimeout.cpp:171"
13:string"folly::EventBase::loopBody at /build/hhvm/third-party/folly/src/folly/io/async/EventBase.cpp:394"
14:string"folly::EventBase::loop at /build/hhvm/third-party/folly/src/folly/io/async/EventBase.cpp:312"
15:string"folly::EventBase::loopForever at /build/hhvm/third-party/folly/src/folly/io/async/EventBase.cpp:535"
16:string"facebook::common::mysql_client::AsyncMysqlClient::<lambda()>::operator() at /build/hhvm/third-party/squangle/squangle/mysql_client/AsyncMysqlClient.cpp:80"
17:string"std::__invoke_impl<void, at /usr/include/c++/7/bits/invoke.h:60"
18:string"std::__invoke<facebook::common::mysql_client::AsyncMysqlClient::init()::<lambda()> at /usr/include/c++/7/bits/invoke.h:95"
19:string"std::thread::_Invoker<std::tuple<facebook::common::mysql_client::AsyncMysqlClient::init()::<lambda()> at /usr/include/c++/7/thread:234"
20:string"std::thread::_Invoker<std::tuple<facebook::common::mysql_client::AsyncMysqlClient::init()::<lambda()> at /usr/include/c++/7/thread:243"
21:string"std::thread::_State_impl<std::thread::_Invoker<std::tuple<facebook::common::mysql_client::AsyncMysqlClient::init()::<lambda()> at /usr/include/c++/7/thread:186"
22:string"start_thread at pthread_create.c:463"
23:string"clone at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95"

Additionally noteworthy is our non standard hacklang pool configuration:

new AsyncMysqlConnectionPool(darray[
	'per_key_connection_limit' => 20,
	'idle_timeout_micros' => 4000000,
	'expiration_policy' => "IdleTime",
]);

URL link broken

https://github.com/facebook/squangle/blob/master/squangle/mysql_client/README.md

The document had a link, but that link not exists

https://our.intern.facebook.com/intern/dex/mysql-client-c-api/

Lack of makefile...

There doesn't seem to be any sign of any makefile or related infrastructure....

[ RFC ] Formatting qualified names with %N (like %C, but allow intermediate dots)

This RFC depends on hhvm's AsyncMysqlConnection->queryAsync() method.

The function appendColumnTableName() in mysql_client/Query.cpp does a runtime check on the QueryArgument passed. If it is a string, it places backticks around it and escapes any actual backticks by tossing in an extra backtick. If a twoTuple is passed, each element of the tuple is escaped this way and they are concatenated using a .. The threeTuple variant interprets the third element as an alias.
Hhvm and Hack have a concept of a tuple(string, string). These tuple types do not match twoTuple and threeTuple. This makes this logic unreachable from hhvm. This means that %C can never be transformed into

`tablename`.`columnname` or `tablename`.`columnname` AS `alias`

Passing the string "t.c"` to %C renders as

`t.c` instead of `t`.`c`

This means you can not pass a qualified name, since the database parser trips over qualified names that have the dot in the backticked context. I do not remiss the aliased threeTuple, but not having a way to passing a qualified name hurts. Hack code can use %T.%C and pass two string arguments. Code that intends to operate with both qualified and unqualified names has to fall back to using %Q.

I would like to be able to use a formatting specifier with works with both kinds of names. This could be performed by tossing in a backtick before and after a . in a string. Adding this to %C will transform a query which fails to parse into one that parses. This may not be what we want if callers expected untrusted arguments to not be able to reach ambiguous fields (join queries with overlapping columns). It also breaks native CPP consumers which pass tuples. Can we introduce a new specifier (I was thinking %N for name) which only operates on strings and tosses in backticks around dots? We might want to throw if the string has more than two parts to prevent injections of database level qualified names (dbname.tablename.columnname).