weso / utils

Common Scala utilities used by several weso projects

Utils

Scala utilities

Introduction

This project contains some common utilities in Scala.

Installation and compilation

The project uses sbt for compilation as well as Java 1.8.

• sbt test compiles and runs the tests

Author & contributors

• Author: Jose Emilio Labra Gayo

Contributors:

• Eric Prud'hommeaux
• Bogdan Roman
• Toni Cebrían
• Andrew Berezovskyi

Adopters

• RDFShape: An online demo powered by this library.
• Wikishape: An online demo powered by this library for Wikidata.

Contribution

Contributions are greatly appreciated. Please fork this repository and open a pull request to add more features or submit issues

Publishing to OSS-Sonatype

This project uses the sbt ci release plugin for publishing to OSS Sonatype.

SNAPSHOT Releases

Open a PR and merge it to watch the CI release a -SNAPSHOT version

Full Library Releases

1. Increment the version number in version.sbt to the desired release version number.
2. Commit and push this change with a message like "blah blah version 0.x.x".
3. Push a tag and watch the CI do a regular release
4. git tag -a v0.1.0 -m "v0.1.0"
5. git push origin v0.1.0 Note that the tag version MUST start with v.

utils's Issues

Remove dependency on Xerces

Hello,

Dependency on Xerces in Shaclex prevents us from compiling under JDK 17:

[INFO] Running org.eclipse.lyo.validation.ShaclMaxExclusiveValidationTest
java.lang.IllegalAccessError: class es.weso.utils.RegEx (in unnamed module @0x1e7ba8d9) cannot access class com.sun.org.apache.xerces.internal.impl.xpath.regex.RegularExpression (in module java.xml) because module java.xml does not export com.sun.org.apache.xerces.internal.impl.xpath.regex to unnamed module @0x1e7ba8d9
	at es.weso.utils.RegEx.matches(RegexUtils.scala:17)
    ...

While not a blocker (we intend to stay on JDK 8 as long as possible for our users), we'd like to resolve build errors in order not to accrue tech debt. Xerces should not be included in the new projects as JAXB has almost everything you need. Moreover, I noticed you only use it for regexp processing. Is there a reason java.util.regex.Pattern cannot be used?

Best regards,
Andrew

cross compile to ScalaJS

Cross compile and publish artifacts to ScalaJS also

Codecov integration not working

Since bb4d033 codecov is raising an error and not processing coverage correctly.

If we go to the coverage webpage of the project we can see that for 79bb81b the coverage was build and proccessed. But the next commit that is receives, bb4d033, was not properly processed.

• 79bb81b was about changes in sutils
• bb4d033 was about added tests

This issue must be fixed prior #4.

Publish scaladoc

Publish scaladoc of the library

Publish API and documentation in gh-pages

Follow the document project to deploy API documentation with asciidoctor + scalaDoc to gh-pages

Create some automated documentation

Create some documentation of the project and integrate it with scaladoc.

Some systems that can be tried:

• Microsite: it seems to provide all what is needed and it is already been used by cats and circe

Enable cross versions so it can be compiled with Scala 2.12 and 2.13 at least

Add repository license

This repo, from creation, remains with no Licence. Similar repos in the organization have MIT / Apache-2 license.

Enable compilation and scoverage for scala 2.13.1

Compiling with scala 2.13.1 worked but when we enable scoverage, it gave this error.

I tried to use scalac-scoverage-plugin but it seems version 1.4.1 is not yet published and it gave me an error.

A simple solution was to revert to scala 2.13.0, but we added this issue so that as it is repaired, we could move forward ti scala 2.13.1.

Meta Issue

Roadmap

The Utils repository contains several small utilities that can be used across several projects.

cross compile to scala 3 - dotty

Check which modules can be cross compiled to scala 3.

Libraries that seem to fail:

• circe
• fs2
• pprint

Libraries that may be removed if they are not needed

• scalatest
• scalactic
• scala-library
• scala-collection-compat
• diffson-circe

Automate the publication of binary artifacts from CI

The publication of artifacts could be automated from the continuous integration process to avoid manual errors.

One important aspect is to remove coverage code before publishing by activating coverageOff.

Enable openjdk for 2.13.1 again

For some reason, openjdk fails to compile for scala 2.13.1. This is the build that failed.

Separate Document to a standalone module or repository

The Document code could be separated to a standalone module and even a repository.

Some tests should be added to increase coverage.

Towards removing this repo

I wonder if this repo is really necessary. At the end, the repo is mostly a "catch-all" repo which contains different utilities for different purposes.

I think this is really an anti-pattern and will try to split the module into other modules/components which have a more clear name and goal.

In this issue we will keep track of the roadmap.

• collectioncompat: will contain the es.weso.internal.utils.CollectionCompat object
• typing: Contains a generic Typing type class, it can be extracted to its own repo
• validating: contains a generic Checker trait that has been implemented in Cats. It was originally implemented using the Eff library. We will remove this module and create a new checker repo.
• utilsTest: this module includes JsonMatchers which are no longer used. We may remove the whole module.
• testSuite: this module could be extracted as a single repo

The utils folder contains the following packages:

• json: contains several utilities to work with Json entities in Scala like DecoderUtils, JsonCompare, JsonTest and JsonTwirl. We may just create a JsonWeso repo for those utilities
• EitherUtils contains two helper methods to work with Either values, sequence and takeSingle. I am not even sure if we use them anymore.
• FileUtils contains some helper methods to work with files: getContents, getFileFromFolderWithExt, writeFile, formatLines, etc.
• ...

Increase coverage to 90%

At this moment the coverage is 33% because most of the code was tested as part of other libraries from which it has been separated. Add more tests to increase coverage.

Runtime dependency on scoverage

I think we are being hit by this issue: Runtime dependency on scoverage when we publish the utils package.

As an example, this build fails.

Little explanation for function fmt at Document

Hi, could you please provide me some information about what this function functionality is ?

utils/modules/utils/src/main/scala/es/weso/document/Document.scala

Lines 66 to 93 in 9cbf59a

	def fmt(k: Int, state: List[FmtState]): Unit = state match {
	case List() => ()
	case (_, _, DocNil) :: z =>
	fmt(k, z)
	case (i, b, DocCons(h, t)) :: z =>
	fmt(k, (i, b, h) :: (i, b, t) :: z)
	case (i, _, DocText(t)) :: z =>
	writer write t
	fmt(k + t.length(), z)
	case (i, b, DocNest(ii, d)) :: z =>
	fmt(k, (i + ii, b, d) :: z)
	case (i, true, DocBreak) :: z =>
	writer write "\n"
	spaces(i)
	fmt(i, z)
	case (i, false, DocBreak) :: z =>
	writer write " "
	fmt(k + 1, z)
	case (i, b, DocGroup(d)) :: z =>
	val fitsFlat = fits(width - k, (i, false, d) :: z)
	fmt(k, (i, !fitsFlat, d) :: z)
	case _ =>
	()
	}
	fmt(0, (0, false, DocGroup(this)) :: Nil)
	}
	}