atk4 / dsql

Object-Oriented SQL Query Builder

Agile Toolkit - Web UI Toolkit

This is LEGACY version of Agile Toolkit. For a brand-new refactored alternative see http://agiletoolkit.org/

Agile Toolkit is a Web UI framework and collection of usable widgets. It allows you to develop rich web applications by writing only PHP code. Agile Toolkit is inspired by Desktop GUI Toolkits and is a fully-object oriented development environment.

• Homepage: http://agiletoolkit.org
• Forums: http://forum.agiletoolkit.org
• Docs: http://book.agiletoolkit.org

Overview

Agile Toolkit has introduced three new principles in web development:

• A Complete UI solution for PHP developers
• Unique integration between jQuery events and chains and PHP
• Innovative Object Relational Manager with support for joins, sub-selects and expression abstraction.

All the features are delivered in a unique close-coupled environment - Similarly to Cocoa or Qt - all objects of Agile Toolkit are based off one common ancestor and are constructed under the guidance of the top-level Application object.

Installing

To start a new web application in Agile Toolkit, download a bundle from http://agiletoolkit.org/ and follow instructions. Alternatively, if you are a very hardcore developer, add the following code inside your composer.json file:

"require": {
    "atk4/atk4": "4.3.*@dev"
}

Example

To help you understand some key principles of Agile Toolkit, copy the following example into page/index.php and place inside the init() method.

Source:

$form = $this->add('Form');
$form->addField('line', 'subject')->validateNotNull();
$form->addField('password','password');
$form->addSubmit();

if ($form->isSubmitted()) {
    $this->js()->univ()
        ->dialogOK('Hello World','Subject: '.$form['subject'])
        ->execute();
}

Congratulations. You have now created a fully AJAX / PHP form, fully protected from SQL / HTML / JS injection, based on jQuery UI theme and Bootstrap-compatible 12-column flexible grid system.

License

Agile Toolkit is distributed under MIT License.

Your support will ensure the longevity of Agile Toolkit

dsql's Issues

Make sure we use UTF8

See http://php.net/manual/en/mysqlinfo.concepts.charset.php

I think at this point in time creating non-UTF8 connections might not be even viable. Previously we used set NAMES UTF8, but now we can simply pass this as an argument.

if table is defined as Expression (not Query), then it's not wrapped in parenthesis

_consume wraps only Query objects in parenthesis and that's OK.

But in case we use Expression object as table it should be wrapped in parenthesis too. See union example in QueryTest class.

Now generates:

select * from (select * from `sales`) union (select * from `purchases`)

should generate:

select * from ((select * from `sales`) union (select * from `purchases`)) `derrivedTable`

derrivedTable alias will come from fixing #33

Multi-field join

The library currently doesn't seem to support multi field joins, or is there already a way to perform them?

I was thinging something like

SELECT A.*
FROM my_tbl_a A
JOIN my_tbl_b B ON (A.x = B.y AND A.b = B.b)
WHERE ...

simpler way to compare two fields [discussion]

Currently when you need to compare two fields, you would normally type:

$q->where('name','surname';)

But this won't compare two fields, instead it will look for records where name="surname". The second try you would do:

$q->where('name', $q->expr('surname'));

But once again, it's not ideal:

where `name` = surname

It's not using quotes now. The third attempt is:

$q->where('name', $q->expr($q->_escape('surname')));

And unfortunately this won't work, because _escape is protected method.

Implement result set

methods getOne(), getRow(), getHash() as well as ability to iterate through result set. Must remain compatible with atk4/atk4.

Implement Expressionable trait (for extending)

It's used in Expression::_consume().
Maybe it should better be interface not trait?

replace is_null with ===null where possible.

Because - performance and I have left some is_null() around:

http://stackoverflow.com/a/8229059/204819

might need to cross-link docs

we might need to cross-link it.

for example
http://dsql.readthedocs.org/en/latest/quickstart.html
basic concepts here
they can be clickable.

http://dsql.readthedocs.org/en/latest/quickstart.html#creating-objects-and-pdo
classes are menitoned here - Query and Expression
again same thing.

http://dsql.readthedocs.org/en/latest/quickstart.html#fetching-result
some methods are mentioned here, but not documented either.
getOne, getRow
i think they nee to be defined here
http://dsql.readthedocs.org/en/latest/queries.html#query-modes

Review code to see if we need to do anything special for UTF8 support.

Check if we have any of these methods and if so,then use their mb_ equivalent.
http://php.net/manual/en/ref.mbstring.php

Also we need to add some test-scripts:

• is utf8 good as a value?
• is utf8 good when dumping queries (debug)
• is utf8 for field names?

Create test-case for something really complex

Based on : https://github.com/atk4/dsql/wiki/Awesome-Queries

Create test-cases for complex queries as explained in Wiki.

• https://github.com/atk4/dsql/wiki/Awesome-Queries#selecting-from-sub-query
• https://github.com/atk4/dsql/wiki/Awesome-Queries#avoiding-and--or-hell
• https://github.com/atk4/dsql/wiki/Awesome-Queries#complex-join
• https://github.com/atk4/dsql/wiki/Awesome-Queries#implementing-if-expressions
• https://github.com/atk4/dsql/wiki/Awesome-Queries#symfony-query-builder-conditions
• https://github.com/atk4/dsql/wiki/Awesome-Queries#composing-sub-queries-for-where-in-condition

Old entries (possibly duplicates)

• http://stackoverflow.com/questions/1925408/handling-complex-where-clauses-with-a-php-query-builder
• http://stackoverflow.com/questions/33568626/laravel-php-complex-conditional-clause-in-query-builder

code duplication in Query:_render_* methods

This piece of code is found in _render_where, _render_orwhere, _render_andwhere, _render_having:

        if (!isset($this->args['where'])) {
            return;
        }

It can me moved to __render_where method to not duplicate code:

        if (!isset($this->args[$kind])) {
            return;
        }

and in previously mentioned methods do simply like this:

    protected function _render_orwhere()
    {
        $ret = $this->__render_where('where');
        return $ret ? implode(' or ', $ret) : null;
    }

Wrong person in your license :)

Looks like you accidentally copy/pasted my name and information into the license in your documentation: https://github.com/atk4/dsql/blob/develop/docs/overview.rst#license

Integrate SQLite and MySQL testing into Travis

Currently we have few tests that run on an actual database. Those must be included in the Travis checks, but travis needs to be configured with MySQL support too.

Update links from "cheat-sheet" to documentation.

I've created a cheat-sheet here:

https://github.com/atk4/dsql#documentation-cheat-sheet

to contain most used functions and link directly into documentation. I'd like someone to help me by simply updating the link to the doc.

Implement "group by"

Simple implementation of group by is also needed.

Implement {} support in expressions

Currently expression support [] arguments and they will consume/parametrize them. We need {} argument support which will consume/escape parameters.

Create dsql-primer project

DSQL has a lot of flexibility, but it's quite difficult to demonstrate with documentation alone.

The idea is to start a new project, that will consist of a very basic PHP files. When you execute those files, they will perform a query and fetch result from a "employee" data-set.

This will make it much easier for new users to learn DSQL.

add __toString() support

Old DSQL would execute getOne() on the query if you attempt to cast it into string. We should preserve this behaviour as it was widely used.

add support for options

should be able to

insert ignore
insert delayed
select calc_found_rows

etc

Possible bug in Query::dsql() returning new self()

The method has the following signature and body:

public function dsql($properties = [])
{
	$q = new self($properties);
	$q->connection = $this->connection;

	return $q;
}

This meas that, when called from an object that is actually a Query_MySQL, it'll return a simple Query instead, shouldn't it be better to use new static() instead ?

Put together a basic website or Release Notes

Before going public and telling others about our cute new library, we should have some
sort of library.

Add PostgreSQL test-scripts

Use the file https://github.com/atk4/dsql/blob/develop/phpunit-mysql.xml as example and create integration for PostgreSQL. Also follow #23 to integrate PostgreSQL into travis.

non-existant SqlFormatter class

If you try to call Expression->getDebugQuery and don't have SqlFormatter class, then you can get ugly exception from tookit because class_exists() (here https://github.com/atk4/dsql/blob/develop/src/Expression.php#L467) actually calls spl_autoload_call which calls toolkit locatePath etc. and if it can not find SqlFormatter.php then it fails :(

Stacktrace:

Exception_PathFinder: File not found

Additional information:

file: SqlFormatter.php
type: php
attempted_locations:
0: \oms\app/lib/SqlFormatter.php
1: \oms\vendor\atk4\atk4/lib/SqlFormatter.php
class: SqlFormatter
namespace:
orig_class: SqlFormatter
\oms\vendor\atk4\atk4\lib\PathFinder.php:461
File    Object Name Stack Trace Help
\oms\vendor\atk4\atk4\lib/PathFinder.php:348    oms_pathfinder  PathFinder->exception("File not found") 
\oms\vendor\atk4\atk4\lib\App/CLI.php:380   oms_pathfinder  PathFinder->locate("php", "SqlFormatter.php", "path")   
\oms\vendor\atk4\atk4\lib/PathFinder.php:461    oms App->locatePath("php", "SqlFormatter.php")  
\oms\vendor\atk4\atk4\lib/PathFinder.php:121    oms_pathfinder  PathFinder->loadClass("SqlFormatter")   
/:  oms_pathfinder  PathFinder->{closure}("SqlFormatter")   
/:  BaseException   BaseExceptionspl_autoload_call("SqlFormatter")   
\oms\vendor\atk4\dsql\src/Expression.php:467    BaseException   BaseExceptionclass_exists("SqlFormatter")    
\oms\vendor\atk4\dsql\src/Expression.php:559    atk4\dsql\Query_MySQL   atk4\dsql\Query_MySQL->getDebugQuery()   
\oms\vendor\atk4\dsql\src/Query.php:1015    atk4\dsql\Query_MySQL   atk4\dsql\Query_MySQL->execute()     
\oms\vendor\atk4\data\src/Join_SQL.php:235  atk4\dsql\Query_MySQL   atk4\dsql\Query_MySQL->delete()  
/:  __p atk4\data\Join_SQL->doDelete(Object(Model_Payment), "6")

...

refactor to use Exception from atk4/core

because, why keep exceptions in both locations.. ?

Achieve code coverage of 90% minimum for 1.0.0 release

Please wait for 1.0-beta before fully resolving this issue.

Execute unbuffered queries

As described here:

http://php.net/manual/en/mysqlinfo.concepts.buffering.php

It is possible to execute unbuffered queries with PDO. We need to make sure that DSQL can still allow it's user to execute unbuffered queries, where the results are streamed, but this mode should be optional.

We also should look in the situations, when we want to execute query while data are still being fetched. Can PDO properly handle that?

Resolve readthedoc auto-build

Currently it complains about non-existant "hook" in github. Maybe the hook can be added manually?

Empty string should always be saved as NULL out-of-the-box

See: http://forum.agiletoolkit.org/t/save-null-value-to-mysql-db/172/2

I strongly believe that empty string should never be saved in DB as such. Empty string is the same as NULL in many DB engines already and have no meaning other than NULL.
So DSQL probably should convert empty string to NULL out-of-the-box.

table() doesn't save alias when table is defined as Expression

Test case:

$q = (new Query)->table('employee', 'emp');
$this->assertEquals(
            'select `name` from (select * from `employee` `emp`) `derrived`',
            (new Query)
                ->field('name')
                ->table($q, 'derrived') // <-- this alias is not saved in $args array if table is expression
                ->render()
        );

Currently this generates

select `name` from (select * from `employee` `emp`)

but should be

select `name` from (select * from `employee` `emp`) `derrived`

rolling back nested tranactions

Question: suppose you have a transaction that adds a new user, that performs multiple operations in DB. This operation calls another routine that adds a new bank account, which also uses a transaction for multiple operations.

Now suppose bank transaction addition operation fails some checks and you want to roll it back. Unfortunately you can't because SQL does not support multiple BEGIN TRANSACTIONS. So the code should somehow throw an exception but I'm not sure how.

add space before 'left join' in qureies

render_query does not add blank space before 'left'. This does not create any problem with queries, but it looks weird:

select `pi`.`name` from `item` `pi`left join `item2` as `pi_i` on `pi_i`.`item_id` = `pi`.`id` where `pi`.`id` = `_i`.`parent_item_id`

Sequential test-scripts.

Some test-scripts are needed to test query in multiple scenarios:

• Fetch all results, then fetch them again.
• use getRow() then fetch results by iterating
• perform 2 queries and fetch them side-by-side.

This needs to be implemented in tests/db

Add support for CASE expression

Currently in order to include CASE expression in your field list, one would have to write:

$query->field(new Expression("case {} when [] then {} when [] then {} else null end", [
    'status',
    'New', 't2.expose_new'
    'Used', 't2.expose_used'
]), 'expose');

http://stackoverflow.com/questions/8600671/mysql-select-statement-with-case-or-if-elseif-not-sure-how-to-get-the-result

Similarly how we are tackling orExpr() we can also add support for caseExpr() which should have a more pleasant format:

$query->field($query->caseExpr('status')
  ->when('New', 't2.expose_new')
  ->when('Used', 't2.expose_used')
  ->else(null), 'expose');

Implement INSERT, UPDATE, DELETE basics

Those should now be easily doable. Create implementation and necessary tests.

Implement misc. expressions, count(), sum(), fx()

Ability easily create expressions off the query.

Multi-connection tests

See if we can run query with multiple connections. Technically it's possible to specify $connection into Expression::execute(), but could it result in problems?

Add join() support

Implement orExpr(), expr(), andExpr()

This now can be done more efficiently since Expression is forked out.

Add partition support into queries

http://dev.mysql.com/doc/refman/5.7/en/delete.html supports "partition (p0)" syntax. We might as well add it or document how to add it.

Implement where() and _render_where() methods

If possible, include "having()" also.

Skip "and" and "or" support (render_or_where)

rename selectTemplate() into mode()

That is more suitable and easier to understand.

Integrate a proper SQL highlighter

https://github.com/jdorn/sql-formatter

Replace Query::templates with Query::template_select, template_update etc.

Improve code climate

https://codeclimate.com/github/atk4/dsql/src/Query.php

Need to improve issues in this file.

Liquidate Query->main_table property

Query->main_table property probably is not needed. At least while we don't have join implemented.

Think of it as - SQL query as such doesn't have "default table" so most likely Query class shouldn't have default (main) table also.

We should properly support tables and fields with spaces e.g. `my table`.

http://stackoverflow.com/a/23773839/204819
http://stackoverflow.com/questions/18184614/create-a-table-with-a-space-in-the-name-in-mysql
http://stackoverflow.com/questions/506826/selecting-a-database-in-mysql-with-spaces-in-its-name

add documentation for order() and limit()

add documentation for order() and limit()
http://dsql.readthedocs.org/en/latest/queries.html#Query::where
only table, field and where is documented.

Add support for WITH clause

Add support in the Query class to implement https://en.wikipedia.org/wiki/Hierarchical_and_recursive_queries_in_SQL#Common_table_expression using SQL WITH clause.

The signature of the function could be the following:

public function with($expr, $table_alias, $fields = [])
public function withRecursive($expr, $table_alias, $fields = [])

The alias in this case should be mandatory (single identifier or array), while expr could be any kind of subtable expression, or even a union. $fields is optional.

To produce the following query

WITH RECURSIVE
  cnt(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM cnt WHERE x<1000000)
SELECT x FROM cnt;

you would do:

$expr = newExpression('values(1) union all select x+1 from cnt where x<1000000');

$q = new Query();
$q->table('cnt');
$q->field('x');
$q->withRecursive($expr, 'cnt', ['x']);

It should be possible to execute with() multiple times.

group(" foo(a, b) ") is being half-escaped

Query::group() detects comma in the argument and will break

$a->group("date_format(dat, '%Y')");

breaks the line into 2 grouping parameters and escapes only the second one resulting in:

group by date_format(dat, "%Y")

implement reset($tag)

as described in queries.rst document