nicmx / rdap-sql-provider Goto Github PK

Let's take this query at CountryCode.sql as an example:

SELECT * FROM {schema}.country_code;

So far there's no problem, but if I do this (there's no space after "*" nor before "FROM"):

SELECT *
FROM {schema}.country_code;

...bad things will happen when the query is executed. The issue is at mx.nic.rdap.sql.QueryGroup constructor, since every line of the query that is being read is concatenated to a StringBuilder; so for the example, the query that is being executed is "SELECT *FROM {schema}.country_code;" which is invalid SQL.

When an user executes a query searching nameservers by name, if the query parameter is a ldhname, the rdap server doesn't bring any results. For example the following rdap query:

http://localhost:8080/rdap-server/nameservers?name=ns1.xn--foo21-pm4d.org

doesn't return any results (despite the fact that there is a valid ns with that name in BD) and it logs the following db query:

SELECT DISTINCT(nse.nse_id), nse.nse_handle,nse.nse_ldh_name, nse.nse_port43, nse.nse_unicode_name FROM rdap.nameserver nse WHERE nse.nse_ldh_name='ns1.fooア21.org' OR nse.nse_unicode_name='ns1.fooア21.org' ORDER BY 1 LIMIT 6;

As commented in rdap-sql-provider issue 13 , the iad_type value is not loaded from database when the whole ip_address object is queried because it is only used when a partial search is executed, it is not used in any setter or getter.

It is recommended to explain this situation on the database script, web-site and code documentation, and on the column description.

The default values of the properties:

• zones
• is_reverse_ipv4_enabled
• is_reverse_ipv6_enabled
• nameserver_as_domain_attribute

are loaded from code (see mx.nic.rdap.sql.model.ZoneModel) instead of being loaded from a configuration file. The values are expected to be at server file WEB-INF/data-access.properties, but they should be at META-INF/sql_provider_configuration.properties; if the implementer wishes to modify the values this could be done at the server file WEB-INF/data-access.properties.

Somehow this has been already discussed, since the comment at mx.nic.rdap.sql.DatabaseSession.testDatabase() line 34 proves that. Instead of hardcode the query, why not let it be configurable?

There's already a configuration file loaded at loadDataSourceFromProperties, my proposal is that this file could have the validation query and just in case that it isn't configured it could load the default query; something like:

mx.nic.rdap.sql.DatabaseSession

/** The parameter "testQuery" is added to the function **/
private static void testDatabase(BasicDataSource ds, String testQuery) throws SQLException {
	try (Connection connection = ds.getConnection(); Statement statement = connection.createStatement();) {
		ResultSet resultSet = statement.executeQuery(testQuery);
		if (!resultSet.next()) {
			throw new SQLException("'" + testQuery+ "' returned no rows.");
		}
		int result = resultSet.getInt(1);
		if (result != 1) {
			throw new SQLException("'" + testQuery+ "' returned " + result);
		}
	}
}
...
private static DataSource loadDataSourceFromProperties(Properties config) throws InitializationException {
...
	/** And is loaded here **/
	// http://stackoverflow.com/questions/3668506
	String testQuery = config.getProperty("testQuery", "select 1");

	try {
		testDatabase(dataSource, testQuery);
	} catch (SQLException e) {
		throw new InitializationException("The database connection test yielded failure.", e);
	}
...
}

The issue was originally reported at the server (rdap-server issue 35), but this issue is for the provider.

When a partial search is made using the only wildcard allowed "*", I still can use other wildcards to get more information in the response. Here are some cases assuming that the minimum of characters allowed for the search is 5:
"/domains?name=%%a%*" will search for all the domains that have the char 'a' in their names.
"/domains?name=__e_*" will search for all the domains that the 3rd char of its name is the char 'e' followed by at least another char.

This wildcards can be used because the queries used for the searches use the sentence "LIKE", so they should be escaped if they are going to reach the query executed at DB. The escape could be made at the models (mx.nic.rdap.sql.model) just when the wildcard "*" is replaced by "%". The issue applies only for extended searches, so the models where the solution can be applied are:

• DomainModel
• EntityModel
• NameserverModel

This issue applies for the services:

• /domain
• /domains?name
• /domains?nsLdhName
• /nameserver
• /nameservers?name

In most of the services the label (domain name, nameserver and zone) is searched just as requested to the service, eg. "/domain/fOo.BaR" will search for the label "fOo" and zone "BaR". The issue is at 2 levels: code and queries.

At code because the values are never sanitized to use them as expected, something similar to the issue reported at the server (rdap-server issue 42) where the zone isn't case insensitive.

At queries because the implementation assumes that the parameters will be just as needed, so if my database stores the domains in uppercase I'm assuming that in dom_ldh_name=? the parameter will already be uppercase to find a match.

My suggestion is that at both levels (code and queries) the values will be used as lowercase, or if possible use operators that can ignore the case.

Some of the queries use the asterisk wildcard to get all the columns from a table, just to say a few:

• CountryCode.sql -> #getAll
• Domain.sql -> #getByLdhName, #searchByPartialNameWZone, #searchByNameWZone, ...
• Entity.sql -> #getByHandle, #getByDomain, #getAll, ...

This doesn't represent a problem (apparently), but it's a good practice and also a recommendation to improve performance, to list only the columns that are expected to be used. In this case, this columns are used by the classes mx.nic.rdap.sql.objects.*DbObj. So I recommend that instead of using the wildcard, the queries should return the columns expected by the java objects.

When a ip address is loaded from database (according to mx.nic.rdap.sql.objects.IpAddressDbObj.java class), all the data is setted with db values, except the internet address type as it is showed in the following method loadFromDatabase:

public void loadFromDatabase(ResultSet resultSet) throws SQLException {
this.setId(resultSet.getLong("iad_id"));
this.setNameserverId(resultSet.getLong("nse_id"));
try {
this.setAddress(InetAddress.getByName(resultSet.getString("iad_value")));
} catch (UnknownHostException e) {
// The address is validate in the store process,so this exception
// will never be throw.
}
if (this.getAddress() instanceof Inet4Address) {
this.setType(4);
} else if (this.getAddress() instanceof Inet6Address) {
this.setType(6);
}

}

Ip Address type attribute is setted in a validation of the instance of inet address, but the iad_type value from db is never accessed. It is necessary to obtain the ip address type from db and, if it is necessary, validate it with the instance of inet address, in order to confirm or throw an exception.

Migrated from NICMx/rdap-server#54 (courtesy of @TheRedTrainer):

Should rdap-server allow to configure an IDN variant TLD as a zone?

I think that it would be useful to allow to configure any kind of TLD as a zone. The current configuration.properties and db table (rdap.zone) allow to configure only one name/representation per zone (for example .com, .lat, .mx).

If you want to configure a TLD that includes a unicode char (for example eú) as a zone in db and configuration.properties, the server allows to configure only the u-label (eú) and query a domain including that unicode char, but there is no column in db or any property to configure the corresponding ldh name.

Some references to objects or to the word "role" are still used as "rol". Here's a list of them:

• mx.nic.rdap.sql.model.RoleModel
  • All the queries names, except "getMainEntityRole" (eg, getDomainRol, getEntityRol, ...)
  • All the search functions (eg. getDomainEntityRol, getNameserverEntityRol, ...)
• Role.sql
  • All the queries identifiers, except "getMainEntityRole".

When a user queries a nameserver, the rdap server retrieves all the information except the ip addresses. It is necessary to provide all the nameserver info including ip addresses

This happens in the object mx.nic.rdap.sql.impl.DomainDAOImpl at the functions searchByName() and searchByNsLDHName(). Both of them have a similar code block:

if (domainName.contains("*")) {
	List<String> labels = Arrays.asList(domainName.split("\\."));
	for (String label : labels) {
		if (label.contains("*") && !label.endsWith("*"))
			throw new BadRequestException("Patterns can only have an * at the end.");
	}
}

So, if the server is allowing the use of wildcards anywhere, this code is preventing to use the wildcard as expected. Technically this validation is done at rdap-server mx.nic.rdap.server.RdapSearchRequest.validatePartialSearchValue(String valuePattern, boolean isEntity).

The column "eve_date" from rdap.event table (a table that contains the information about events that have occurred on an instance of an object class) is defined as VARCHAR(45). However, if an user fill this field with a random string that doesn't fit with any date format, and the server execute a query, the following error is displayed in server's logs:

10-Oct-2017 15:53:58.786 SEVERE [http-nio-8080-exec-2] mx.nic.rdap.server.servlet.RdapServlet.doGet java.sql.SQLException: Bad format for Timestamp 'PATATA PATATA 2016-11-13 00:09:20' in column 4.
mx.nic.rdap.db.exception.RdapDataAccessException: java.sql.SQLException: Bad format for Timestamp 'PATATA PATATA 2016-11-13 00:09:20' in column 4.
at mx.nic.rdap.sql.impl.DomainDAOImpl.searchByName(DomainDAOImpl.java:70)
at mx.nic.rdap.server.servlet.DomainSearchServlet.getPartialSearch(DomainSearchServlet.java:96)
at mx.nic.rdap.server.servlet.DomainSearchServlet.doRdapDaGet(DomainSearchServlet.java:67)
at mx.nic.rdap.server.servlet.DomainSearchServlet.doRdapDaGet(DomainSearchServlet.java:22)
at mx.nic.rdap.server.servlet.DataAccessServlet.doRdapGet(DataAccessServlet.java:38)
at mx.nic.rdap.server.servlet.RdapServlet.doGet(RdapServlet.java:34)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
...
Caused by: java.sql.SQLException: Bad format for Timestamp 'PATATA PATATA 2016-11-13 00:09:20' in column 4.
at com.mysql.jdbc.ResultSetRow.getTimestampFast(ResultSetRow.java:1157)
at com.mysql.jdbc.ByteArrayRow.getTimestampFast(ByteArrayRow.java:130)
at com.mysql.jdbc.ResultSetImpl.getTimestampInternal(ResultSetImpl.java:5946)
at com.mysql.jdbc.ResultSetImpl.getTimestamp(ResultSetImpl.java:5616)
at com.mysql.jdbc.ResultSetImpl.getTimestamp(ResultSetImpl.java:5645)
mainDAOImpl.searchByName(DomainDAOImpl.java:65)...
... 31 more

It is necessary to treat the eve_date as a date, in order to avoid any issue when parsing the string to a date format.

The dom_ldh_name and dom_unicode_name columns from domain table has a length of 64 and 255 chars respectively. A FQDN (Fully Qualified Domain Name) has a limit of 63 chars, so it is necessary to fix the max length for these columns in order to comply the rdap operational profile.

ine_start_address_down and ine_end_address_down columns from ip_network are currently nullable values. However, these columns shouldn't allow null values because they contain critical data for ip network such as ip address (at least the last octets/segments).

If a client search a domain or a nameserver by unicode name that includes a char with a similar unicode value (for example á ~ a) the db return all the coincidences for the char. If a client want to retrieve data from domain "exámple.com", the query returns all the coincidences that matches similar unicode chars (example.com, éxample.com, exámple.com, examplé.com, éxámplé.com), beside the fact that the ldh_name for these domains are different because áa, ée have differente unicode values.

The query should search for the exact unicode char, at leats in domain unicode name search.

The handle column for domain, entity and nameserver table is commonly used as a container for RIR unique identifier, and it is defined as a not nullable field for ip network and ASN objects.

However, the handle column is a nullable field for domain, entity and nameserver tables. Should it be managed as nullable?

If a rdap server needs to implement entities credentials (for registrant, administrative and/or technical users), the current rdap_user table structure doesn't allow it because the user name column (rus_name) can contains only 16 chars, instead of the 100 char length for entity name column ent_handle.

It is required to match the length for both columns in order to allow to any entity/contact being a rdap user too.

Almost all the columns for LINK table have a length of 45 chars (lin_value, lin_rel, lin_href, lin_title, lin_media, lin_type).

Should all these columns have that length? Some data, such as href, could contain a domain name or email address that could exceed this limit.