jep-data-extension

jep-data-extension is a project that extends JEP, the powerful mathematical expression parser and evaluator for Java, introducing data manipulation capabilities with custom functions and operators ready to use, including RegEx, XPath, JSONPath and Web Services.

For details about JEP core features, access JEP documentation.

New functions

Package	Name	Description
String functions	camel	Converts a string to camel case
	concat	Concatenates the arguments into a string.
	endsWith	Returns 1 (true) if a string ends with the specified suffix.
	findMatch	Returns the first match of the given regular expression found inside a string.
	findMatches	Returns a list containing all matches of the given regular expression inside a string.
	formatString	Returns a formatted string according to given pattern and variable arguments.
	lpad/leftPad	Left-pads a string to a given size.
	lcase/lower	Converts a text string to all lower-case letters.
	matches	Returns a 1 if the given string contains at least one match of a given RegEx.
	normalizeString	Normalizes a Unicode string, replacing accents and other diacritics with ASCII chars.
	proper	Converts a string to proper case, i.e., only the first letter in each word in upper-case.
	replace	Replaces all occurrences of a given string with another one.
	replaceRegex	Replaces all matches of a given regular expression.
	rpad/rightPad	Right-pads a string to a given size.
	split	Splits a string into a string array based on a separating string or regular expression.
	startsWith	Returns 1 if a string starts a given prefix.
	trim	Removes leading and trailing spaces.
	ucase/upper	Converts a text string to all upper-case letters.
Date functions	now/sysdate	Returns the system's current date & time.
	str2date	Converts a string into date by trying different patterns such as RFC-3339, RFC-822 and common ISO-8601 variations.
	date2str	Converts a date into string using a specified format.
	daysBetween	Returns the number of days between two dates.
	endOfMonth	Returns a date corresponding to the last day of the month given a source date.
	isLeapYear	Returns 1 if the given date or year number is a leap year, i.e., an year with 366 days.
	year	Returns the year for a given date.
	quarter	Returns the quarter of the year for a given date.
	month	Returns the month for a given date, a number starting from 1 (Jan) to 12 (Dec).
	isoWeekNumber	Returns the ISO week number for a given date.
	weekday	Returns the day of the week of a date, a number from 1 (Sunday) to 7 (Saturday).
	day	Returns the day of the month for a given date..
	hour	Returns the hour of day for a given date, as a number from 0 (12AM) to 23 (11PM).
	minute	Returns the minute within the hour for a given date, as a number from 0 to 59.
	second	Returns the seconds within the minute for a given date, a number from 0 to 59.
	addDays	Returns the result from adding a number of days to a date.
Math functions	arabic	Converts a Roman numeral to Arabic.
Math functions	roman	Converts an Arabic numeral to Roman.
Data manipulation & filtering	xpath	Returns the result of the given XPath expression at the specified XML.
Data manipulation & filtering	jsonpath	Returns the result of the given JSONPath expression at the specified JSON.
Statistical functions	average	Returns the average of the elements inside a given array, JSON array or collection (including valid representations).
	count	Returns the number of elements inside a given array, JSON array or collection (including valid string representations).
	max	Returns the largest value in a given array, JSON array or collection (including valid string representations).
	min	Returns the smallest value in a given array, JSON array or collection (including valid string representations).
Random	uuid	Produces a randomly generated type 4 UUID (Universally-unique Identifier) string.
Utility functions	distinct	Returns a list consisting of the distinct elements of a given list, array or JSON array.
	getEnv	Returns the value of an environment variable.
	getSystemProperty	Returns the value of system property.
	isDecimal	Returns 1 if the given parameter is a number (including strings) containing decimals.
	isEmpty	Returns 1 if a given parameter is either null or an empty String, JSON or collection.
	isInteger	Returns 1 if the given number (or valid string) represents a whole number, i.e, which lacks decimals.
	readFile	Returns the content of a text file in the file system.
	typeOf/class	Returns the Java type associated with the given variable.
Cryptography functions	sha256	Computes the SHA-256 hash of the given string and transforms the binary result into a hexadecimal string.
	toBase64	Encodes a string to Base64 encoding scheme.
	fromBase64	Decodes a Base64 encoded string.
Web Services	httpGet	Returns data from a Web Service or RESTful API, as string.
	http	Invokes a specific method towards a Web Service/REST API.
	httpHeader	This function groups a variable number of HTTP header entries for usage with other HTTP functions.
	basicAuthorizationHeader	Generates a basic authorization header from a given username and password.
	httpStatusCode	Returns the HTTP status code of a given WebServiceResponse.
	httpResponse	Returns the HTTP response body from given WebServiceResponse, as string.

ℹ️ Find examples in the wiki.

Custom operators

JEP offers a good set of operators for common arithmetic and logical operations, for example: addition (+), subtraction (-), multiplication (*), division (/), modulus (%), power (^), "Boolean And" (&&), "Boolean Or" (||), and "Boolean Not" (!). These operators can be used when required and are supported by JEP.

Comparative operators

Standard JEP relational operators accept only numerical variables (including string representation of non-complex, numerical values). These operators have been overloaded to allow comparison of dates, as well as string representations of valid dates:

Less than (<)
Less than or equal to (<=)
Greater than (>)
Greater than or equal to (<=)
Equal (==)
Not equal (!=)

This allows the usage of dates in expressions that can be evaluated to true or false. For example:

if("2017-03-11T10:15:00:123Z" < now(), "past", "not past")

Element operator

Standard "element" operator ([]) was overloaded to support data extraction out of JSON arrays and other collections by index.

jsonArray[1] //extracts the first element of the given jsonArray

Note: Also achievable using the function get(jsonArray, 1).

How to include it

If you are using Maven, add jep-data-extension as a dependency to your pom.xml file.

<dependency>
    <groupId>net.obvj</groupId>
    <artifactId>jep-data-extension</artifactId>
    <version>1.0.7</version>
</dependency>

If you use other dependency managers (such as Gradle, Grape, Ivy, etc.) click here.

How to use it

Example 1: replacing characters from a text variable using the `JEPContextFactory`

Add jep-data-extensions to your class path

Put your source variables in a map:

Map<String, Object> myVariables = new HashMap<>();
myVariables.put("myText", "foo");

Use the JEPContextFactory class to create an extended JEP object with all available extensions:
```
JEP jep = JEPContextFactory.newContext(myVariables);
```
Note: Alternatively, you may use the JEPContextFactory.newContext() to receive a new context with no initial variables. You may add them later, calling jep.addVariable(String, Object).

Parse your expression:

Node node = jep.parseExpresion("replace(myText, \"oo\", \"ee\")");

Evaluate the expression with JEP's evaluate method:

String result = (String) jep.evaluate(node); //result = "fee"

Alternatively, you can use the assignment operator inside an expression. The new variable will be available in JEP context for reuse and can be retrieved using the getVarValue method:
```
Node node = jep.parseExpression("myText = replace(\"fear\", \"f\", \"d\")"));
jep.evaluate(node);
String result = (String) jep.getVarValue("myText"); //result = "dear"
```

Example 2: comparing dates using the `ExpressionEvaluatorFacade`

The ExpressionEvaluatorFacade is a convenient object to quickly parse a single expression. A single call to the evaluate method will create a new evaluation context and return the expression result directly. It accepts a map with initial variables for use in the operation.

Put your source variables in a map:

Map<String, Object> myVariables = new HashMap<>();
myVariables.put("date1", "2018-12-20T09:10:00.123456789Z");

Evaluate your expression:

String expression = "if(date1 < now(), \"overdue\", \"not overdue\")";
String result = (String) ExpressionEvaluatorFacade.evaluate(expression, myVariables); 
//result = "overdue"

Example 3: Consuming data from a RESTful API using the Evaluation Console

Run jep-data-extension JAR file from a terminal:
```
$ java -jar jep-data-extension-1.0.2.jar
```

Consume the RESTful API with the httpGet function:

JEP > dateTime = httpGet("http://date.jsontest.com/")
{
   "time": "08:31:19 PM",
   "date": "05-12-2012"
}

Filter the JSON using a JsonPath expression that extracts the date field:
```
JEP > date = jsonpath(dateTime, "$.date")
05-12-2012
```
Check the type of the generated date variable:
```
JEP > class(date)
java.lang.String
```

Convert the string to Date:

JEP > date = str2date(date, "dd-MM-yyyy")
Wed Dec 05 00:00:00 BRST 2012

Extract the week number from the converted date:
```
JEP > isoWeekNumber(date)
49
```

Introducing named packages for optimization

Enhancement

Proposal

Currently, a call to JEPContextFactory.newContext() provides a JEP object with all 110 functions available (standard and custom).
Also, JEP does not support the concept of packages. So, a feasible choice for optimizing the context creation would be informing the JEPContextFactory to create a new context with a limited set of functions.

Basically, the proposal is to allow an option to specify one or more packages (as varargs) which functions should to be loaded into JEP's function table.

Examples of usage:

 // all functions available, as usual (Core JEP + ALL custom functions)
JEP jepWithAllFunctions = JEPContextFactory.newContext();

// Core JEP functions + custom string functions (concat, trim, rpad, etc.)
JEP jepWithStringFunctions = JEPContextFactory.newContext(NamedPackage.STRING);

// Core JEP functions + custom string and web-service functions (concat, trim, rpad, httpGet, etc.)
JEP jepWithStringFunctions = JEPContextFactory.newContext(NamedPackage.STRING, NamedPackage.WEB_SERVICES);

Additional details

The code shall be developed in the issue/0010 branch.

Quality criteria

The following criteria will be observed during the Code Review:

Code coverage

The produced code shall be secured by unit tests.
Test coverage shall remain the same or increase.

Documentation

All methods shall be documented with Javadoc, providing examples of usage.
Javadoc pages shall be compilable with no errors or warnings
README.md shall be updated if applicable

oswaldobapvicjr / jep-data-extension Goto Github PK