GithubHelp home page GithubHelp logo

apache / shardingsphere Goto Github PK

View Code? Open in Web Editor NEW
19.4K 1.0K 6.6K 619.8 MB

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.

License: Apache License 2.0

Java 95.08% Shell 0.06% Batchfile 0.01% Dockerfile 0.02% ANTLR 4.33% FreeMarker 0.49%
database distributed-database distributed-sql-database distributed-transactions sql shard database-cluster mysql postgresql database-plus encrypt dba rdbms oltp bigdata hacktoberfest

shardingsphere's Introduction

Official Website: https://shardingsphere.apache.org/

GitHub Release Lines of Code

Quality Gate Status Technical Debt Maintainability Rating Security Rating codecov

OpenSSF Best Practices

Slack Gitter

Twitter

Stargazers Over Time Contributors Over Time
Stargazers Over Time Contributor over time

OVERVIEW


Apache ShardingSphere is a distributed SQL transaction & query engine that allows for data sharding, scaling, encryption, and more - on any database. Our community's guiding development concept is Database Plus for creating a complete ecosystem that allows you to transform any database into a distributed database system.

It focuses on repurposing existing databases, by placing a standardized upper layer above existing and fragmented databases, rather than creating a new database.

The goal is to provide unified database services and minimize or eliminate the challenges caused by underlying databases' fragmentation. This results in applications only needing to communicate with a single standardized service.

The concepts at the core of the project are Connect, Enhance and Pluggable.

  • Connect: Flexible adaptation of database protocol, SQL dialect and database storage. It can quickly connect applications and heterogeneous databases.
  • Enhance: Capture database access entry to provide additional features transparently, such as: redirect (sharding, readwrite-splitting and shadow), transform (data encrypt and mask), authentication (security, audit and authority), governance (circuit breaker and access limitation and analyze, QoS and observability).
  • Pluggable: Leveraging the micro kernel and 3 layers pluggable mode, features and database ecosystem can be embedded flexibly. Developers can customize their ShardingSphere just like building with LEGO blocks.

ShardingSphere became an Apache Top-Level Project on April 16, 2020.

So far, ShardingSphere has been used by over 10,000 projects on GitHub.

DOCUMENTATION📜


EN doc CN doc

For full documentation & more details, visit: Docs

CONTRIBUTION🚀🧑💻


For guides on how to get started and setup your environment, contributor & committer guides, visit: Contribution Guidelines

Team


We deeply appreciate community contributors for their dedication to Apache ShardingSphere.

COMMUNITY & SUPPORT💝🖤


🔗 Mailing List. Best for: Apache community updates, releases, changes.

🔗 GitHub Issues. Best for: larger systemic questions/bug reports or anything development related.

🔗 GitHub Discussions. Best for: technical questions & support, requesting new features, proposing new features.

🔗 Slack channel. Best for: instant communications and online meetings, sharing your applications.

🔗 Twitter. Best for: keeping up to date on everything ShardingSphere.

🔗 LinkedIn. Best for: professional networking and career development with other ShardingSphere contributors.

STATUS👀


✅ Version 5.4.1: released 🎉

🔗 For the release notes, follow this link to the relevant GitHub page.

🔜 Version 5.4.2

We are currently working towards our 5.4.2 milestone. Keep an eye on the milestones page of this repo to stay up to date.

How it Works


Apache ShardingSphere includes 2 independent products: JDBC & Proxy. They all provide functions of data scale-out, distributed transaction and distributed governance, applicable in a variety of situations such as Java-based isomorphism, heterogeneous language and Cloud-Native.

ShardingSphere-JDBC


Maven Status

A lightweight Java framework providing extra services at the Java JDBC layer. With the client end connecting directly to the database, it provides services in the form of a jar and requires no extra deployment and dependence.

🔗 For more details, follow this link to the official website.

ShardingSphere-Proxy


Nightly-Download Download Docker Pulls

A transparent database proxy, providing a database server that encapsulates the database binary protocol to support heterogeneous languages. Friendlier to DBAs, the MySQL and PostgreSQL version now provided can use any kind of terminal.

🔗 For more details, follow this link to the official website.

Hybrid Architecture


ShardingSphere-JDBC adopts a decentralized architecture, applicable to high-performance light-weight OLTP applications developed with Java. ShardingSphere-Proxy provides static entry and all languages support, suitable for an OLAP application and sharding databases management and operation.

Through the combination of ShardingSphere-JDBC & ShardingSphere-Proxy together with a unified sharding strategy by the same registry center, the ShardingSphere ecosystem can build an application system suitable to all kinds of scenarios.

🔗 More details can be found following this link to the official website.

Solution


Solutions/Features Distributed Database Data Security Database Gateway Stress Testing
Data Sharding Data Encryption Heterogeneous Databases Supported Shadow Database
Read/write Splitting Row Authority (TODO) SQL Dialect Translate (TODO) Observability
Distributed Transaction SQL Audit (TODO)
Elastic Scale-out SQL Firewall (TODO)
High Availability

Roadmap


Roadmap

How to Build Apache ShardingSphere


Check out Wiki section for details on how to build Apache ShardingSphere and a full guide on how to get started and setup your local dev environment.

Landscapes




  

Apache ShardingSphere enriches the CNCF CLOUD NATIVE Landscape.

shardingsphere's People

Contributors

azexcy avatar beckhampu avatar cherrylzhao avatar codefairy08 avatar dongzl avatar flyingzc avatar haocao avatar jingshanglu avatar kimmking avatar komachision avatar lanchengx avatar maxiaoguang64 avatar menghaoranss avatar pace2car avatar qianyi951015 avatar raigorjiang avatar saaavsaaa avatar sandynz avatar soulasuna avatar strongduanmu avatar taojintianxia avatar terrymanu avatar teslacn avatar totalo avatar tristazero avatar tuichenchuxin avatar tuohai666 avatar yu199195 avatar yx9o avatar zhaojinchao95 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

shardingsphere's Issues

代码编译不过

public final class DataNode {

private final String dataSourceName;

private final String tableName;

}

这个代码IDE过不了,而且也没有意义,是不是?

OR表达式下会出现重复结果问题

假设有两个分表:tb0, tb1; 按照id字段取模。现在有where查询条件 id = 0 or id = 2,则导致在分表tb0上执行两次这样的条件查询,出现重复结果: <0,2>, <0, 2>

改造多线程执行模型

原来的实现是每次执行SQL时新建线程池,这样做不够高效。
改造为每个ShardingDataSource对象维护一个线程池。

consistency when executing update

Hi:
In Sharding-jdbc PreparedStatementExecutor will execute RoutedPreparedStatement list concurrently. Do you have any mechanism to ensure the consistency between multiple datasources when executing updates? It seems to me that if the connection is set auto-commit, one datasource' failure will not rollback other datasources' execution and therefore the overall consistency is not guaranteed. Do we have to use manual commit when using sharding-jdbc to do update operation?

动态表配置

可以不配置真实表和逻辑表的对应关系,通过分片算法动态计算真实表。

动态表因为预先不知道真实表和逻辑表的对应关系,所以以下功能不支持:

  1. 不带分片键的全路由
  2. BindingTable

MySQLSelectVisitor与druid最新G.A版不兼容

Hi, 当前DRUID的版本最新是1.0.17,而sharding-jdbc使用的是1.0.12,而更新编译后发现MySqlSelectGroupByExpr 这个类已经从druid中移除掉了,这个该如解决(我们一些环境的DRUID版本均是1.0.16等版本,考虑到工程依赖于druid的sql parser处理);另外,shard-jdbc可否考虑增加针对oracle的支持(计划什么时候增加相关支持)

必须根据where条件找到切分字段?如果一个子条件没有找到,貌似行为不确定,默认不支持查询全部库

// 如果这里没有找到切分字段,也就没有了ConditionContext集
if (!shardingColumns.contains(column.getColumnName())) {
return;
}

// 没有了ConditionContext,这里也就为空,后面就不了了之了,
for (SQLExecutionUnit each : sqlRouteResult.getExecutionUnits()) {
PreparedStatement preparedStatement = generatePrepareStatement(getShardingConnection().getConnection(each.getDataSource()), each.getSql());
replayMethodsInvovation(preparedStatement);
setParameters(preparedStatement, parameters);
result.add(preparedStatement);
}

如果默认不支持全部分片的查询,建议要抛个异常

向含有text字段类型的分表中插入数据时失败

MyBatis+MySQL,向含有text字段类型的分表中插入数据数据时失败,异常信息如下:

Caused by: java.sql.SQLException: Invalid argument value: java.io.NotSerializableException
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:998)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:937)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:872)
at com.mysql.jdbc.PreparedStatement.setSerializableObject(PreparedStatement.java:3887)
at com.mysql.jdbc.PreparedStatement.setObject(PreparedStatement.java:3603)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_setObject(FilterChainImpl.java:2923)
at com.alibaba.druid.filter.FilterAdapter.preparedStatement_setObject(FilterAdapter.java:1298)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_setObject(FilterChainImpl.java:2920)
at com.alibaba.druid.filter.FilterAdapter.preparedStatement_setObject(FilterAdapter.java:1298)
at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_setObject(FilterChainImpl.java:2920)
at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.setObject(PreparedStatementProxyImpl.java:411)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.setObject(DruidPooledPreparedStatement.java:476)
at com.dangdang.ddframe.rdb.sharding.jdbc.ShardingPreparedStatement.setParameters(ShardingPreparedStatement.java:188)
at com.dangdang.ddframe.rdb.sharding.jdbc.ShardingPreparedStatement.routeSQL(ShardingPreparedStatement.java:163)
at com.dangdang.ddframe.rdb.sharding.jdbc.ShardingPreparedStatement.routeIfNeed(ShardingPreparedStatement.java:151)
at com.dangdang.ddframe.rdb.sharding.jdbc.ShardingPreparedStatement.getRoutedPreparedStatements(ShardingPreparedStatement.java:138)
at com.dangdang.ddframe.rdb.sharding.jdbc.ShardingPreparedStatement.execute(ShardingPreparedStatement.java:107)

Caused by: java.io.NotSerializableException: java.io.StringReader
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at com.mysql.jdbc.PreparedStatement.setSerializableObject(PreparedStatement.java:3876)

sharding insert简单测试异常提示?

Hi, 做sharding-jdbc单元测试的时候遇到了提示

com.dangdang.ddframe.rdb.sharding.exception.SQLParserException: Sharding columns DO NOT exist in insert column names
at com.dangdang.ddframe.rdb.sharding.parser.visitor.basic.mysql.MySQLInsertVisitor.visit(MySQLInsertVisitor.java:45)

我这边的单元测试(构建shardingRule)时没有配置DatabaseShardingStrategy或TableShardingStrategy
——经测试select、update以及delete均没有问题

代码为:
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.sql.DataSource;

import org.junit.Test;

import com.dangdang.ddframe.rdb.sharding.api.DatabaseType;
import com.dangdang.ddframe.rdb.sharding.api.rule.DataSourceRule;
import com.dangdang.ddframe.rdb.sharding.api.rule.ShardingRule;
import com.dangdang.ddframe.rdb.sharding.api.rule.TableRule;
import com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit;
import com.dangdang.ddframe.rdb.sharding.router.SQLRouteEngine;
import com.dangdang.ddframe.rdb.sharding.router.SQLRouteResult;

public class TestShardingRule {

@Test
public void testShardingRule1() {
    ShardingRule sr1 = createShardingRule1();

    // 构建routerEngine
    SQLRouteEngine routerEngine = new SQLRouteEngine(sr1, DatabaseType.MySQL);

// String testQuerySQL = "select o.column1, o.column2 from t_order o where o.cloumn3 = 'aa'";
//
// SQLRouteResult routerResult1 = routerEngine.route(testQuerySQL, Collections.emptyList());
//
// List executionUnitList1 = routerResult1.getExecutionUnits();
//
// for (SQLExecutionUnit executionUnit : executionUnitList1) {
// System.out.println("query sql unit:" + executionUnit.getSql());
// }

    String testInsertSQL = "insert into t_order(column1, column2, column3) values('aa', 'bb', 'cc')";

    SQLRouteResult routerResult2 = routerEngine.route(testInsertSQL, Collections.emptyList());

    List<SQLExecutionUnit> executionUnitList2 = routerResult2.getExecutionUnits();

    for (SQLExecutionUnit executionUnit : executionUnitList2) {
        System.out.println("insert unit sql:" + executionUnit.getSql());
    }

// String testUpdateSQL = "update t_order t set t.column1 = 'aa' where t.column2 = 'bb' and t.column3 = 'cc'";
//
// SQLRouteResult routerResult3 = routerEngine.route(testUpdateSQL, Collections.emptyList());
//
// List executionUnitList3 = routerResult3.getExecutionUnits();
//
// for (SQLExecutionUnit executionUnit : executionUnitList3) {
// System.out.println("update unit sql:" + executionUnit.getSql());
// }

// String testDeleteSQL = "delete from t_order t where t.column2 = 'bb' and t.column3 = 'cc'";
//
// SQLRouteResult routerResult4 = routerEngine.route(testDeleteSQL, Collections.emptyList());
//
// List executionUnitList4 = routerResult4.getExecutionUnits();
//
// for (SQLExecutionUnit executionUnit : executionUnitList4) {
// System.out.println("delete unit sql:" + executionUnit.getSql());
// }
}

private ShardingRule createShardingRule1() {
    return new ShardingRule(createDataSourceRule(), Arrays.asList(createTableRule()));
}

private DataSourceRule createDataSourceRule() {
    Map<String, DataSource> result = new HashMap<>(2);
    result.put("ds0", null);
    result.put("ds1", null);
    return new DataSourceRule(result);
}

private TableRule createTableRule() {
    return new TableRule("t_order", Arrays.asList("t_order_0", "t_order_1", "t_order_2"),
            createDataSourceRule());
}

}

我想确认的是,是否insert的操作必须要指明Sharding columns麽

ORDER BY与GROUP BY衍生列未添加表名或表别名

  1. order by 生成的补充列不带所有表的别名
    bug现象
select ... order_id AS sharding_gen_1 from order a order by a.order_id

应该为

select ... a.order_id AS sharding_gen_1 from order a order by a.order_id
  1. group by 生成补充列时没有判断查询投射中是否已经存在了该列
    bug现象
select a.order_id, order_id AS sharding_gen_1 from order a group by a.order_id

应该为

select a.order_id  from order a order by a.order_id

为什么OR条件下不进行聚合函数处理

if (sqlVisitor.getParseContext().isHasOrCondition()) {
result = new OrParser(sqlStatement, visitor).parse(); // 这里直接丢掉了聚合列的merge处理?
} else {
sqlVisitor.getParseContext().mergeCurrentConditionContext();
result = sqlVisitor.getParseContext().getParsedResult();
}

count获取分表当中的所有数量有问题

分库分表情况
db1: order_0(记录数33)、order_1(记录数34)、order_2(记录数33)
db2: order_0(记录数0)、order_1(记录数0)、order_2(记录数0)

执行语句:
select count(1) from order 结果:33
执行语句:
select count(*) as c from order 结果:100

ShardingPreparedStatement.execute()方法中缓存语句cachedRoutedPreparedStatements未更新导致的问题

String sql = "insert into user_info_shard(id,user_no,user_name) values(?,?,?)";
try {
Connection conn = shardingDS.getConnection();
PreparedStatement pstmt = conn.prepareStatement(sql);

                for (int i = 1; i <= 10000; i++) {
                    pstmt.setInt(1, i);
                    pstmt.setString(2, UUID.randomUUID().toString().replace("-", ""));
                    pstmt.setString(3,"ZHANG"+i);

                    pstmt.execute();
                }


        } catch(Exception ex) {
            ex.printStackTrace();
        }

单库、分表测试,只有一张逻辑表。
测试代码如上,该测试代码只有第一条记录插入成功,第二条记录插入时就出异常(主键冲突),原因是ShardingPreparedStatement的参数重新设置以后,cachedRoutedPreparedStatements未更新。

sharding-jdbc-config-common编译问题

@hanahmily, 编译1.1.0-SNAPSHOT标题组件的时候,*.common.internal 下的ConfigUtil类一直编译不通过。经定位是GROOVY包没有引用正确的缘故,由于indy一直无法正常导入,更新替代成下面则可编译通过

org.codehaus.groovy
groovy-all

2.4.6

——注释掉classifiier,将artifactid调整为groovy-all并注明其版本(如2.4.6)
我对groovy不了解,考虑到ALL的包比较大(应该不需要All,但indy方式不清楚无法下载依赖JAR参与编译),可否说明一下(若上述存在问题,也希望能指出)

sbt添加 "com.dangdang" % "sharding-jdbc-core" % "1.0.0"出错

[info] Resolving com.fasterxml.jackson.datatype#jackson-datatype-jsr310;2.5.3 ..[info] Resolving com.fasterxml.jackson.datatype#jackson-datatype-jsr310;2.5.4 ..[info] Resolving org.hibernate.javax.persistence#hibernate-jpa-2.1-api;1.0.0.Fin[info] Resolving net.contentobjects.jnotify#jnotify;0.94-play-1 ...
[error] impossible to get artifacts when data has not been loaded. IvyNode = org.apache.commons#commons-lang3;3.3.2
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.apache.commons#commons-lang3;3.3.2
at org.apache.ivy.core.resolve.IvyNode.getArtifacts(IvyNode.java:809)
at org.apache.ivy.core.resolve.IvyNode.getSelectedArtifacts(IvyNode.java:786)
at org.apache.ivy.core.report.ResolveReport.setDependencies(ResolveReport.java:235)
at org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:235)
at org.apache.ivy.Ivy.resolve(Ivy.java:517)
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:279)
at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:188)
at sbt.IvyActions$$anonfun$updateEither$1.apply(IvyActions.scala:165)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:155)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:132)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:57)
at sbt.IvySbt$$anon$4.call(Ivy.scala:65)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:78)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:97)
at xsbt.boot.Using$.withResource(Using.scala:10)
at xsbt.boot.Using$.apply(Using.scala:9)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:58)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:48)
at xsbt.boot.Locks$.apply0(Locks.scala:31)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at sbt.IvySbt.withDefaultLogger(Ivy.scala:65)
at sbt.IvySbt.withIvy(Ivy.scala:127)
at sbt.IvySbt.withIvy(Ivy.scala:124)
at sbt.IvySbt$Module.withModule(Ivy.scala:155)
at sbt.IvyActions$.updateEither(IvyActions.scala:165)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1369)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1365)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1399)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$87.apply(Defaults.scala:1397)
at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:37)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1402)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1396)
at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:60)
at sbt.Classpaths$.cachedUpdate(Defaults.scala:1419)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1348)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1310)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
error java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = org.apache.commons#commons-lang3;3.3.2
[error] Total time: 3 s, completed Feb 29, 2016 12:43:56 AM

对最大努力重试的疑问

sharding-jdbc支持的两种柔性事务TCC和最大努力, 我理解一个是反向补偿,一个是正向重试,后者假设2个参与者,1个成功,一个失败,失败的哪个如果是逻辑冲突如主键重复了,怎么重试都会失败,基础设施问题,重启后恢复了,可能重试成功,但间隔较大,这期间告诉客户端事务是中间状态?
TCC 2个参与者,1个成功,一个失败,会立即都反向补偿,如果是逻辑原因的,可理解补偿。如果是基础设施,好的那个参与者可以补偿。也就是最大努力下逻辑错误,重试也不行,基础设施故障能重试成功取决于基础设施恢复要多久,TCC下,逻辑错误,基础实施故障都能回退。那么最大努力正向重试除了对应用透明,还有何优势而言呢?

代码风格问题

1、代码大量使用Lombok,导致eclipse找不到对应方法
2、好多长语句
这两点导致代码看着有点累。。
不过还得硬着头皮看下去

API调整,抽离ShardingDataSource,使用工厂代替

com.dangdang.ddframe.rdb.sharding.api.ShardingDataSource变更为com.dangdang.ddframe.rdb.sharding.jdbc.ShardingDataSource,原com.dangdang.ddframe.rdb.sharding.api.ShardingDataSource 已废弃,未来版本将删除。
推荐使用ShardingDataSourceFactory创建DataSource。

查询Count时,使用getObject()取数会报异常

现象:
SQL: select count(*) as cnt from tbl;

如果我使用 ResultSet.getLong()getInt()都可以。
但是如果我使用 getObject(),就会报异常:
ShardingJdbcException("Unsupported data type:%s", convertType)

原因:
s-jdbc在聚合Merge时,会根据调用方法的返回值来转换merge的结果,当调用getObject()Number类型结果时,就需要将Number转为Object了,现在s-jdbc直接报不支持的类型。

修改方法:
对于返回类型为Object时,直接将Number值返回。

代码位置:
com/dangdang/ddframe/rdb/sharding/merger/common/ResultSetUtil.java
convertNumberValue方法:
我给增加了

case "java.lang.Object":
              return value;

在我本地测试没有问题了。

count函数在某些情况下返回不正确

参考示例代码里的场景,求t_order的count总数:

使用mybatis,分成3个库,每个库里分2张表,每个表里插入2条记录,正确的count值应该是12

    <select id="queryCount" resultType="int">
        select count(*) from t_order
    </select>

仅返回一个int型整数,返回的数值,仅为某条子查询的count值,没有累加,下面是输出的日志:

15:00:28,341 <com.dangdang.ddframe.rdb.sharding.parser.SQLParserFactory> DEBUG [main]: Logic SQL: select count(0) from t_order
15:00:28,391 <com.dangdang.ddframe.rdb.sharding.parser.SQLParserFactory> TRACE [main]: Get com.alibaba.druid.sql.ast.statement.SQLSelectStatement SQL Statement
15:00:28,783 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelectStatement
15:00:28,822 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 visit argument: SELECT COUNT(0)
FROM t_order
15:00:28,822 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelect
15:00:28,823 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 visit argument: com.alibaba.druid.sql.ast.statement.SQLSelect@1d60737e
15:00:28,823 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 visit node: class com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock
15:00:28,823 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 visit argument: com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock@c7242101
15:00:28,856 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelectItem
15:00:28,856 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 visit argument: com.alibaba.druid.sql.ast.expr.SQLAggregateExpr@640d1fb2
15:00:28,856 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 visit node: class com.alibaba.druid.sql.ast.expr.SQLAggregateExpr
15:00:28,856 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 visit argument: com.alibaba.druid.sql.ast.expr.SQLAggregateExpr@640d1fb2
15:00:28,858 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 visit node: class com.alibaba.druid.sql.ast.expr.SQLIntegerExpr
15:00:28,859 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 visit argument: 0
15:00:28,859 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit node: class com.alibaba.druid.sql.ast.expr.SQLIntegerExpr
15:00:28,859 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,860 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit condition: ConditionContext(conditions={})
15:00:28,860 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit SQL: SELECT COUNT(0
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit node: class com.alibaba.druid.sql.ast.expr.SQLAggregateExpr
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit condition: ConditionContext(conditions={})
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit SQL: SELECT COUNT(0)
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelectItem
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit condition: ConditionContext(conditions={})
15:00:28,879 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit SQL: SELECT COUNT(0)
15:00:28,882 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit node: class com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock
15:00:28,882 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,882 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit condition: ConditionContext(conditions={})
15:00:28,885 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit SQL: SELECT COUNT(0) FROM [Token(t_order)]
15:00:28,885 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelect
15:00:28,885 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,885 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit condition: ConditionContext(conditions={})
15:00:28,885 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit SQL: SELECT COUNT(0) FROM [Token(t_order)]
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelectStatement
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit condition: ConditionContext(conditions={})
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit SQL: SELECT COUNT(0) FROM [Token(t_order)]
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.SQLParseEngine> DEBUG [main]: Parsed SQL result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[ConditionContext(conditions={})], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.absent(), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:00:28,886 <com.dangdang.ddframe.rdb.sharding.parser.SQLParseEngine> DEBUG [main]: Parsed SQL: SELECT COUNT(0) FROM [Token(t_order)]
15:00:28,904 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: Before database sharding t_order routes db names: [ds_0, ds_1, ds_2] sharding columns: [user_id] sharding values: []
15:00:28,904 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: After database sharding t_order result: [ds_2, ds_1, ds_0]
15:00:28,904 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: Before table sharding t_order routes db names: [DataNode(dataSourceName=ds_0, tableName=t_order_0), DataNode(dataSourceName=ds_1, tableName=t_order_0), DataNode(dataSourceName=ds_2, tableName=t_order_0), DataNode(dataSourceName=ds_0, tableName=t_order_1), DataNode(dataSourceName=ds_1, tableName=t_order_1), DataNode(dataSourceName=ds_2, tableName=t_order_1)] sharding columns: [order_id] sharding values: []
15:00:28,904 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: After table sharding t_order result: [t_order_0, t_order_1]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_0] sql: [SELECT COUNT(0) FROM t_order_0]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_0] sql: [SELECT COUNT(0) FROM t_order_1]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_1] sql: [SELECT COUNT(0) FROM t_order_0]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_1] sql: [SELECT COUNT(0) FROM t_order_1]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_2] sql: [SELECT COUNT(0) FROM t_order_0]
15:00:28,907 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_2] sql: [SELECT COUNT(0) FROM t_order_1]
15:00:29,084 <com.dangdang.ddframe.rdb.sharding.executor.ExecutorEngine> TRACE [main]: Concurrent execute result success [true, true, true, true, true, true]
15:00:29,088 <com.dangdang.ddframe.rdb.sharding.merger.ResultSetFactory> TRACE [main]: Get 'Aggregate' result set
Feb 22, 2016 3:00:29 PM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext@3d82c5f3: startup date [Mon Feb 22 15:00:24 CST 2016]; root of context hierarchy
count => 2 ***这是我输出的结果

但是,如果改成

    <select id="queryCount" resultType="int">
        select count(*) order_count from t_order
    </select>

写成这样,即:加一个字段别名后,返回结果是正确的,会把各分库、分表子查询的count值累加返回,输出日志如下:

15:03:49,055 <com.dangdang.ddframe.rdb.sharding.parser.SQLParserFactory> DEBUG [main]: Logic SQL: select count(0) order_count from t_order
15:03:49,105 <com.dangdang.ddframe.rdb.sharding.parser.SQLParserFactory> TRACE [main]: Get com.alibaba.druid.sql.ast.statement.SQLSelectStatement SQL Statement
15:03:49,505 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelectStatement
15:03:49,540 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 visit argument: SELECT COUNT(0) AS order_count
FROM t_order
15:03:49,540 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelect
15:03:49,540 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 visit argument: com.alibaba.druid.sql.ast.statement.SQLSelect@1d60737e
15:03:49,540 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 visit node: class com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock
15:03:49,540 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 visit argument: com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock@c7242101
15:03:49,572 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 visit node: class com.alibaba.druid.sql.ast.statement.SQLSelectItem
15:03:49,572 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 visit argument: com.alibaba.druid.sql.ast.expr.SQLAggregateExpr@640d1fb2 AS order_count
15:03:49,572 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 visit node: class com.alibaba.druid.sql.ast.expr.SQLAggregateExpr
15:03:49,572 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 visit argument: com.alibaba.druid.sql.ast.expr.SQLAggregateExpr@640d1fb2
15:03:49,574 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 visit node: class com.alibaba.druid.sql.ast.expr.SQLIntegerExpr
15:03:49,574 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 visit argument: 0
15:03:49,574 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit node: class com.alibaba.druid.sql.ast.expr.SQLIntegerExpr
15:03:49,574 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,575 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit condition: ConditionContext(conditions={})
15:03:49,575 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5   6 endVisit SQL: SELECT COUNT(0
15:03:49,604 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit node: class com.alibaba.druid.sql.ast.expr.SQLAggregateExpr
15:03:49,604 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,604 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit condition: ConditionContext(conditions={})
15:03:49,604 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4   5 endVisit SQL: SELECT COUNT(0)
15:03:49,605 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelectItem
15:03:49,605 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,605 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit condition: ConditionContext(conditions={})
15:03:49,605 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3   4 endVisit SQL: SELECT COUNT(0) AS order_count
15:03:49,606 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit node: class com.alibaba.druid.sql.dialect.mysql.ast.statement.MySqlSelectQueryBlock
15:03:49,606 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,606 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit condition: ConditionContext(conditions={})
15:03:49,609 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2   3 endVisit SQL: SELECT COUNT(0) AS order_count FROM [Token(t_order)]
15:03:49,609 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelect
15:03:49,609 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,609 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit condition: ConditionContext(conditions={})
15:03:49,610 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1   2 endVisit SQL: SELECT COUNT(0) AS order_count FROM [Token(t_order)]
15:03:49,610 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit node: class com.alibaba.druid.sql.ast.statement.SQLSelectStatement
15:03:49,610 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,610 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit condition: ConditionContext(conditions={})
15:03:49,610 <com.dangdang.ddframe.rdb.sharding.parser.visitor.VisitorLogProxy> TRACE [main]:   1 endVisit SQL: SELECT COUNT(0) AS order_count FROM [Token(t_order)]
15:03:49,611 <com.dangdang.ddframe.rdb.sharding.parser.SQLParseEngine> DEBUG [main]: Parsed SQL result: SQLParsedResult(routeContext=RouteContext(tables=[Table(name=t_order, alias=Optional.absent())], sqlBuilder=null), conditionContexts=[ConditionContext(conditions={})], mergeContext=MergeContext(orderByColumns=[], groupByColumns=[], aggregationColumns=[AggregationColumn(expression=COUNT(0), aggregationType=COUNT, alias=Optional.of(order_count), option=Optional.absent(), derivedColumns=[], index=1)], limit=null))
15:03:49,611 <com.dangdang.ddframe.rdb.sharding.parser.SQLParseEngine> DEBUG [main]: Parsed SQL: SELECT COUNT(0) AS order_count FROM [Token(t_order)]
15:03:49,637 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: Before database sharding t_order routes db names: [ds_0, ds_1, ds_2] sharding columns: [user_id] sharding values: []
15:03:49,638 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: After database sharding t_order result: [ds_2, ds_1, ds_0]
15:03:49,638 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: Before table sharding t_order routes db names: [DataNode(dataSourceName=ds_0, tableName=t_order_0), DataNode(dataSourceName=ds_1, tableName=t_order_0), DataNode(dataSourceName=ds_2, tableName=t_order_0), DataNode(dataSourceName=ds_0, tableName=t_order_1), DataNode(dataSourceName=ds_1, tableName=t_order_1), DataNode(dataSourceName=ds_2, tableName=t_order_1)] sharding columns: [order_id] sharding values: []
15:03:49,638 <com.dangdang.ddframe.rdb.sharding.router.single.SingleTableRouter> TRACE [main]: After table sharding t_order result: [t_order_0, t_order_1]
15:03:49,641 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_0] sql: [SELECT COUNT(0) AS order_count FROM t_order_0]
15:03:49,641 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_0] sql: [SELECT COUNT(0) AS order_count FROM t_order_1]
15:03:49,641 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_1] sql: [SELECT COUNT(0) AS order_count FROM t_order_0]
15:03:49,641 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_1] sql: [SELECT COUNT(0) AS order_count FROM t_order_1]
15:03:49,642 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_2] sql: [SELECT COUNT(0) AS order_count FROM t_order_0]
15:03:49,642 <com.dangdang.ddframe.rdb.sharding.router.SQLExecutionUnit> DEBUG [main]: route sql to db: [ds_2] sql: [SELECT COUNT(0) AS order_count FROM t_order_1]
15:03:49,774 <com.dangdang.ddframe.rdb.sharding.executor.ExecutorEngine> TRACE [main]: Concurrent execute result success [true, true, true, true, true, true]
15:03:49,779 <com.dangdang.ddframe.rdb.sharding.merger.ResultSetFactory> TRACE [main]: Get 'Aggregate' result set
15:03:49,826 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 2
15:03:49,826 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 4
15:03:49,827 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 6
15:03:49,827 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 8
15:03:49,827 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 10
15:03:49,827 <com.dangdang.ddframe.rdb.sharding.merger.aggregation.AccumulationAggregationUnit> TRACE [main]: Accumulation result: 12
Feb 22, 2016 3:03:49 PM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext@3d82c5f3: startup date [Mon Feb 22 15:03:45 CST 2016]; root of context hierarchy
count => 12 *** 这次就对了

各位大大,有空看下,谢谢!

调整属性配置

在ShardingDataSource传入properties的配置中,如下修改:

metrics.second.period => metrics.millisecond.period
metrics.package.name => metrics.logger.name
parallelExecutor.worker.minIdleSize => executor.min.idle.size
parallelExecutor.worker.maxSize => executor.max.size
parallelExecutor.worker.minIdleSize => executor.min.idle.size
parallelExecutor.worker.maxIdleTimeout => executor.max.idle.timeout.millisecond

目的:

  1. 规范并简化命名
  2. 统一时间单位为毫秒

精准定位子查询中的补充列行为

由于order by和group by在select 中补充查询列。而对于存在子查询的场景,此种补充列行为会影响内外层sql。
在解析层面对子查询进行识别,保证补充列行为可以进行精准定位。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.