The anshar from entur

False validation errors on valid date in FramedVehicleJourneyRef

Sporadically we get false validation errors of the kind: Value [2023-04-04] is invalid for field [DataFrameRef], expected valid date with PATTERN yyyy-MM-dd. Obviously this is a valid date according to pattern and there is also a comment in the code:

} catch (NumberFormatException e) {
    // Happens sometimes without actually being wrong...
    // Return false to flag for manual follow-up
    return false;
}

A likely cause is that the java.text.Format used in parsing is not thread safe. JDK recommends DateTimeFormatter as an immutable and thread-safe alternative.

There is a an old StackOverflow thread on the topic.

Allow configuring the Siri version of an outgoing subscription request

I'm currently dealing with a Siri server that only allows you to send subscription request when the version is set to 2.0 and nothing else.

So this request works:

<Siri
	xmlns="http://www.siri.org.uk/siri">
	<SubscriptionRequest>
		<RequestTimestamp>2023-04-11T15:16:11.423257Z</RequestTimestamp>
		<RequestorRef>leonard</RequestorRef>
		<MessageIdentifier>bcf4c940-4475-4759-bbfb-6b5457a0be92</MessageIdentifier>
		<ConsumerAddress>https://example.com</ConsumerAddress>
		<SituationExchangeSubscriptionRequest>
			<SubscriberRef>leonard</SubscriberRef>
			<SubscriptionIdentifier>sx</SubscriptionIdentifier>
			<InitialTerminationTime>2023-06-20T15:16:11.424437Z</InitialTerminationTime>
			<SituationExchangeRequest version="2.0">
				<RequestTimestamp>2023-04-11T15:16:11.423992Z</RequestTimestamp>
				<MessageIdentifier>cf9ccbfb-da4c-43eb-a1a2-c52c32fea099</MessageIdentifier>
				<PreviewInterval>PT6H</PreviewInterval>
			</SituationExchangeRequest>
		</SituationExchangeSubscriptionRequest>
	</SubscriptionRequest>
</Siri>

but this one doesn't:

<Siri
	xmlns="http://www.siri.org.uk/siri">
	<SubscriptionRequest>
		<RequestTimestamp>2023-04-11T15:16:11.423257Z</RequestTimestamp>
		<RequestorRef>leonard</RequestorRef>
		<MessageIdentifier>bcf4c940-4475-4759-bbfb-6b5457a0be92</MessageIdentifier>
		<ConsumerAddress>https://example.com</ConsumerAddress>
		<SituationExchangeSubscriptionRequest>
			<SubscriberRef>leonard</SubscriberRef>
			<SubscriptionIdentifier>sx</SubscriptionIdentifier>
			<InitialTerminationTime>2023-06-20T15:16:11.424437Z</InitialTerminationTime>
			<SituationExchangeRequest version="2.1">
				<RequestTimestamp>2023-04-11T15:16:11.423992Z</RequestTimestamp>
				<MessageIdentifier>cf9ccbfb-da4c-43eb-a1a2-c52c32fea099</MessageIdentifier>
				<PreviewInterval>PT6H</PreviewInterval>
			</SituationExchangeRequest>
		</SituationExchangeSubscriptionRequest>
	</SubscriptionRequest>
</Siri>

Since it's hard to spot: the difference is in the line with SituationExchangeRequest. The version is 2.1 but I need it to be 2.0 or nothing.

Anshar hardcodes the version to be 2.1.

What is the reason for this?

Would you allow a PR that uses the version from the subscription to set it there?

Do we even need to set this version? Can we simply remove it?

cc @lassetyr

Use Cases

Hi folks,
Thank you so much for providing and maintaining this repository. I am happy to announce that I am operating a test instance that successfully interacts with Entur's CEN SIRI services.

To move a step forward I am wondering about the use cases of this repository. The following is my understanding so far. Is that correct?

According to the readme Anshar is a SIRI hub.
Subscribed SIRI services (inbound) are forwarded to subscribed clients (outbound).
Incoming XML from SIRI services are validated against the SIRI XSD.

The following URL's from the readme seem to have issues. Are they still up to date?

http://<host>:<port>/up
http://<host>:<port>/ready
http://<host>:<port>/healthy

Can you imagine a reason why I see in the admin-gui a warning triangle and no processed objects for the ET and SX service but not for the SIRI VM service?

Cheers!

Public container image

I would like to start exploring this software and I wondering if you would be interested in having a public container image.

If so, I would like to contribute the following:

a maven based images built using the excellent tool jib which we also use to build container images for OTP
a Github Actions workflow that builds the image on every merge to master and pushes it to Github Container Registry

WDYT?

cc @lassetyr

Cannot read XML message in UTF-16 encoding

When an XML message in UTF-16 arrives from a subscription, anshar throws the following error:

 08:52:44.041 [WARN] (SiriXmlValidator.java:229) Caught exception when parsing
 com.ctc.wstx.exc.WstxParsingException: Declared encoding 'UTF-16' uses 2 bytes per character; but physical encoding appeared to use 1; cannot decode
  at [row,col,system-id]: [1,39,"N/A"]
         at com.ctc.wstx.io.InputBootstrapper.reportXmlProblem(InputBootstrapper.java:489)
         at com.ctc.wstx.io.StreamBootstrapper.verifyEncoding(StreamBootstrapper.java:991)
         at com.ctc.wstx.io.StreamBootstrapper.verifyXmlEncoding(StreamBootstrapper.java:459)
         at com.ctc.wstx.io.StreamBootstrapper.bootstrapInput(StreamBootstrapper.java:175)
         at com.ctc.wstx.stax.WstxInputFactory.doCreateSR(WstxInputFactory.java:577)
         at com.ctc.wstx.stax.WstxInputFactory.createSR(WstxInputFactory.java:637)
         at com.ctc.wstx.stax.WstxInputFactory.createSR(WstxInputFactory.java:651)
         at com.ctc.wstx.stax.WstxInputFactory.createXMLStreamReader(WstxInputFactory.java:338)
         at no.rutebanken.anshar.routes.validation.SiriXmlValidator.parseXml(SiriXmlValidator.java:193)
         at no.rutebanken.anshar.routes.validation.SiriXmlValidator.parseXml(SiriXmlValidator.java:183)
         at no.rutebanken.anshar.routes.messaging.MessagingRoute.lambda$configure$3(MessagingRoute.java:188)
         at org.apache.camel.support.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:65)
         at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:477)
         at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:181)
         at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:59)
         at org.apache.camel.processor.Pipeline.process(Pipeline.java:165)
         at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:392)
         at org.apache.camel.component.jetty.CamelContinuationServlet.doService(CamelContinuationServlet.java:245)
         at org.apache.camel.http.common.CamelServlet.service(CamelServlet.java:130)
         at javax.servlet.http.HttpServlet.service(HttpServlet.java:596)
         at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
         at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
         at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
         at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
         at org.eclipse.jetty.server.Server.handle(Server.java:516)
         at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
         at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
         at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
         at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
         at java.base/java.lang.Thread.run(Unknown Source)
 08:52:44.041 [ERROR] (CamelLogger.java:205) Failed delivery for (MessageId: 5F2578A098A0CEA-0000000000122B53 on ExchangeId: 5F2578A098A0CEA-0000000000122B53). Exhausted after delivery attempt: 1 caught: co>
  at [row,col,system-id]: [1,39,"N/A"]

This is because the String from the HTTP body is converted to BufferedInputStream without checking the encoding.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Warning

Renovate failed to look up the following dependencies: Failed to look up maven package com.google.cloud:pubsub-emulator.

Files affected: pom.xml

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Update dependency net.sf.saxon:Saxon-HE to v12

Detected dependencies

circleci

.circleci/config.yml

cimg/openjdk 17.0.1

cimg/openjdk 17.0.1

cimg/openjdk 17.0.1

github-actions

.github/workflows/ci.yml

actions/checkout v3

actions/setup-java v3

actions/checkout v3

actions/setup-java v3

maven

pom.xml

org.entur.ror:superpom 2.4

org.springframework.boot:spring-boot-dependencies 3.1.5

org.apache.camel.springboot:camel-spring-boot-bom 4.1.0

net.sf.saxon:Saxon-HE 11.5

io.confluent:kafka-avro-serializer 7.4.1

io.confluent:kafka-schema-registry-client 7.4.1

io.confluent:kafka-streams-avro-serde 7.4.1

jakarta.ws.rs:jakarta.ws.rs-api 3.1.0

jakarta.xml.bind:jakarta.xml.bind-api 4.0.0

com.sun.xml.bind:jaxb-core 4.0.4

com.sun.xml.bind:jaxb-impl 4.0.4

org.glassfish.jaxb:jaxb-runtime 4.0.4

com.fasterxml.jackson.dataformat:jackson-dataformat-xml 2.15.3

org.entur:siri-avro-mapper 2.5.10

org.apache.avro:avro 1.11.3

org.entur.ror.helpers:gcp-storage 2.4

org.entur.ror.helpers:hazelcast4-helper 2.4

com.hazelcast:hazelcast 5.1.7

org.entur:siri-java-model 1.27

org.entur:siri-protobuf-mapper 1.0.3

org.entur:netex-java-model 2.0.15

org.entur:netex-parser-java 3.1.4

org.geotools:gt-main 30.0

org.geotools:gt-referencing 30.0

org.apache.commons:commons-lang3 3.13.0

com.googlecode.json-simple:json-simple 1.1.1

javax.el:javax.el-api 3.0.0

com.esotericsoftware:kryo-shaded 4.0.3

xerces:xercesImpl 2.12.2

org.json:json 20231013

com.google.guava:guava 32.1.3-jre

org.apache.camel:camel-test-junit5 4.1.0

org.apache.camel:camel-test-spring-junit5 4.1.0

io.rest-assured:rest-assured 5.3.1

io.rest-assured:json-path 5.3.2

io.rest-assured:xml-path 5.3.1

org.apache.maven.plugins:maven-compiler-plugin 3.11.0

org.apache.maven.plugins:maven-surefire-plugin 3.1.2

org.springframework.boot:spring-boot-maven-plugin 3.1.5

com.google.cloud:pubsub-emulator 1.51.0

com.google.cloud.tools:jib-maven-plugin 3.4.0

org.apache.avro:avro-maven-plugin 1.11.3

Check this box to trigger a request for Renovate to run again on this repository

How to configure Anshar to subscribe CEN SIRI by Entur?

Hi folks,
Thank you so much for providing and maintaining this repository.

I heart it is possible to subscribe a CEN SIRI VM service using the following address.

https://api.entur.io/realtime/v1/subscribe

The following datasets are looking interesting, as they are offered also in the GTFS-RT format.

VYB Vy Buss (SE)
VYX Vy Express

From the browser it is obvious that the ENTUR services are working quite well. I get an HTTP response when I send the following GET request.

https://api.entur.io/realtime/v1/rest/vm

Anyhow, I learned that the subscriptionMode: SUBSCRIBE requires the property anshar.inbound.url to be set. It is set in this file.

So, what is the required value for this property? Do I configure something like this.

# Required for SUBSCRIBE-subscriptions
anshar.inbound.url = https://api.entur.io/realtime/v1/subscribe

Do you mind sharing an example config for Anshar that subscribes a CEN SIRI service like VM?

Cheers!

entur / anshar Goto Github PK

anshar's People

Contributors

Stargazers

Watchers

Forkers

anshar's Issues

Rate-Limited

Open

Ignored or Blocked

Detected dependencies

Recommend Projects

Recommend Topics

Recommend Org

Jobs