open-dis / open-dis-java Goto Github PK

While the answer to this issue might be well duh...

I am working with simulation software that I don't have control over that is regularly sending Data PDUs that contain variable datums greater in the realm of ~2000 bytes.This is causing a buffer underflow exception to originate from \edu\nps\moves\net\BehaviorProducerUDP.java, due to it’s hardcoded buffer size on the ethernet MTU constant in \edu\nps\moves\net\BehaviorWriterIF.

Assuming I am not wildly off base and misunderstanding the situation would it be better for open-dis to allow the user to configure that recieve buffer size if the user is operating in a network that supports UDP fragmentation? The DIS standard itself says the max size of a PDU is 8K bytes (again) if I am not mistaken. Simply changing that constant to a larger MTU size in the open-dis source alleviates my issues but I don't want to deviate.

It looks like you have .jar's for up to 4.16 but the maven dependency is only at 4.08...

The package name for VariableLengthData is wrong.

I guess it's the generator that needs to be fixed, but here's the detail of the correction: #5

We provide DisTime to convert System.currentimemillis to dis time but not a conversion back. Ideally, given a pdu time, get the millis since epoch equivalent. Does this library provide that or is anyone aware of the java code that successfully does the conversion?

I've made some tests with CoordinateConversions.java and conclude that I can't take the reverse results correctly.

Example:

double[] geods = new double[3];
geods[0] = -23.0;
geods[1] = -45.0;
geods[2] = 0.0;

double[] result = CoordinateConversions.getXYZfromLatLonDegrees(geods[0], geods[1], geods[2]);
// result[0] = 4153622.0304340417
// result[1] = -4153622.030434041
// result[2] = -2476719.3300588187

// Lets reverse the result to prove the formula:
double[] proof = CoordinateConversions.xyzToLatLonDegrees( result );

// proof[0] = -23.000000000000004
// proof[1] = -44.99999999999999
// proof[2] = -9.313225746154785E-10

The coordinates error is fine ( a little ) but the elevation is far incorrect and lead to simulation inconsistency ( maybe a collision or ammo hit event).

Is the formula correct?

In the DIS7.xml file, RecordQueryReliablePdu is currently has value="63", but should have value="65".

I personally am not yet up to speed enough on the xmlpg and subsequent code patches to make this update...

My co-worker @MurrayMcDonald16 discovered this one.

If you look at the Articulated Parameters you will see that each ends with a 32-bit float field followed by a 32-bit padding field.

Unfortunately the open-dis ArticulationParameter class marshals/unmarshals the parameter value as a 64-bit double value. This is incorrect.

I am using the following code to get the correct "value" for each ArticulationParameter :

// temporary work-around for open-dis incorrect handling of the parameter value
Double value = articulationParameter.getParameterValue();
ByteBuffer buff = ByteBuffer.allocate(8);
buff.putDouble(value); 
buff.flip();
Float realValue = buff.getFloat();

Thanks,

Murray

Any idea why I'm getting this in the console (infinitely) as a result of a generic exception: "Problems decoding multiple PDUs in datagram; decoded as may as possible"

From line 428 in PduFactory.java (getPdusFromBundle)

https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/PduFactory.java

The implementation I was using (was working until a few days ago, I think something got pushed from our DIS folks)

public List<Pdu> getCurrentBundle() {
        List<Pdu> toReturn = new ArrayList<>();
        LOGGER.debug("getBundle");
        try {
            toReturn = getListFromBuffer();
        } // End try
        catch (Exception e) {
            LOGGER.error(e);
        }

        return toReturn;
   }

    private List<Pdu> getListFromBuffer(){
        DatagramPacket packet;
        PduFactory pduFactory = new PduFactory();
        byte buffer[] = new byte[MAX_PDU_SIZE];
        packet = new DatagramPacket(buffer, buffer.length);
        try {
            socket.receive(packet);
        }catch (Exception e){
            LOGGER.error(e);
        }
        List<Pdu> pduBundle = pduFactory.getPdusFromBundle(packet.getData());
        return  pduBundle;
    }

While reviewing what exists in the edu.nps.moves.disutil package I see a number of classes that appear to serve questionable utility outside niche use cases and perhaps add more noise for newcomers of the library than value. The following list of classes are candidates to be marked deprecated. And then after a period of time would be removed from the project. If you are a user of any of these classes please comment below and explain if you feel the removal of any would be a mistake. Thank-you

1. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/ByteBufferPool.java
2. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/ConcatenatedDisPdus.java
3. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/CoordinateTransformer.java
4. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/DisMarshaller.java
5. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/NioServer.java
6. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/PduMulticastReceiver.java
7. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/PduNioMulticastReceiver.java
8. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/UdpServer.java
9. https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/disutil/VariableLengthData.java

Hi All,
I am not able to locate where the SISO-REF-010 triggers the generation of the OpenDIS enumerations files. The version that currently exists in the C# code base (for example) uses the CET2010 entity types, which are slowly getting deprecated.

Does OpenDIS have the capability to auto-generate C++,C#,or Java files based on a new SISO-REF-010.xml/xsd file set?

If so can anyone point me in the direction of where this auto-generation is performed?

Thanks!

Hi I was wondering when there will be an update to properly support the Variable Parameters on PDUs like the Transmitter PDU? Currently you can get the length, but are unable to get/set these fields. Is there a plan to update this soon or a workaround?

The createPdu method does not currently handle Entity State Update, or Electromagnetic Emissions (a.k.a. Electronic Emissions) PDUs, and possibly others. Recommend updating the switch statement to include cases for all known PDUs.

I've encountered a problem, where my Application identifier can get mangled.

The API for EntityID allows site, application and entity to be stored as int values. But, on marshalling the data it gets converted to a short:

       dos.writeShort( (short)site);
       dos.writeShort( (short)application);
       dos.writeShort( (short)entity);

This will mangle the values if they are greater than 32768.

I was declaring (& sending) an application value of 230318, but my OriginatingEntityId was received as 33710.

I believe the correct fix is to change the storage & getters/setters of site, application and entity so that they handle short values, and correctly reflect the spec:
http://faculty.nps.edu/brutzman/vrtp/mil/navy/nps/disEnumerations/JdbeHtmlFiles/pdu/2c.htm

I noticed that Maven Central only has up to 4.0.8. Any chance of getting newer versions deployed up there for easier integration?

Thanks!

The DIS7 EntityMarking characters field should be an 11 byte array, the same as in the DIS 6 Marking class

Per IEEE 1278.1 (5.8.4.2 in the draft available at http://movesinstitute.org), the Signal PDU data length should be in bits.

edu.nps.moves.dis.SignalPDU is consistent with the standard. However, the data length setter is inconsistent with the preceding comment and could result in a malformed PDU.

edu.nps.moves.dis7.SignalPDU is not consistent with the standard. This is a subtle bug because marshalling and unmarshalling will work "correctly" if all simulation applications are using Open DIS and DIS7 Signal PDUs.

In addition, according 5.8.4.3.2, the data length should be the number of valid bits (i.e. excluding padding). Both edu.nps.moves.dis.SignalPDU and edu.nps.moves.dis7.SignalPDU are not consistent with this specification as well as the padding requirement:

• edu.nps.moves.dis.SignalPDU may lose up to 7 bits of data due to integer division
• edu.nps.moves.dis7.SignalPDU may erroneously include up to 7 random bits of data if the caller does not explicitly zero-pad or mask the final octet

So we have a (wonderful 😄 ) Maven pom.xml and we can do all the cool Maven'y things offers for free from the command line; cleaning, compiling, running unit tests and building the jar, building the javadocs.

$ mvn clean
$ mvn compile
$ mvn test
$ mvn package
$ mvn javadoc:javadoc

etc.

However what we can't do is open the Maven pom.xml project in Netbeans. This is because Netbeans gets confused by the existing nbproject folder which is configured for the Ant project.

As a workaround I've been locally removing the nbproject folder before I attempt to open the project in Netbeans.

Per the comments in the code: " it is coded up, but the linear motion does not seem to work....rotation works but linear motion fails...not sure why they are calculating the linear motion the way they are..."

I have a Java version of the C code in the Towers & Hines paper referenced at the end of IEEE Std 1278.1-2012. My Java version passes the two test cases at the end of that paper. Note that the C code in the Towers & Hines paper (and my Java version of the same) handles all Dead Reckoning algorithms in one function by treating the simpler algorithms as special cases of the more complicated algorithms.

I think it'd be useful to start using Travis CI to run our build and unit tests on every push for us. It can also build any PR branches to ensure we aren't unknowingly merging broken code.

I think the only changes required will be:

• giving Travis CI access to the project to watch for pushes
• creating a .travis.yml config file in the root of the project to tell Travis how to build the project (I'll make a PR for that once I figured that out)

I'm not sure which is the current master copy of java sources, but I have a problem with VariableDatum.java, as recorded in the SVN head here

public void setVariableData(byte[] pVariableData)
{ 
    int unitsOfEight = pVariableData.length / 8;
    int remainder = pVariableData.length % 8;

    if(remainder != 0)
        unitsOfEight++;

    byte[] newData = new byte[unitsOfEight];
    for(int idx = 0; idx < pVariableData.length; idx++)
    {
        newData[idx] = pVariableData[idx];
    }

    variableData = newData;
    variableData = pVariableData;
}

I have two issues. The first is an array index problem in the newData[idx] = call. newData is an array of (roughly) pVariableData.length / 8. But, we're assigning to it using an array loop of pVariableData.length. This causes the array index problem.

The second is that we assign the new data array to variableData, then we assign the raw pVariableData array to variableData, negating our 8-byte array transformation performed earlier in the method.

I guess the intended behaviour is that newData should be an array of 8-byte arrays. If this is correct, then I'm happy to create a PR accordingly. But, my guess could be way off.

Event type 21 (EventReport) isn't being handled in the PduFactory switch (pduTypeEnum) { construct.

I'll submit a PR.

I have a hunch most of the jars in the lib dir are from older examples and from a time when hibernate and jaxb was supported.

I'm thinking even xmlpg could be built as needed from it's source code instead of keeping a copy in there.

If the Maven pom.xml is a good indication of what's really needed, then everything aside from dis-enums, srm and junit 4.12 can be removed.

Looks like there's a missing dependency for disenum.jar in the pom.xml file.

I tried to remove the Hibernate dependencies from the Maven pom.xml, but I got a few compiler errors. It appears there are a few places that still reference either Hibernate annotations or the import javax.persistence.*

• edu.nps.moves.dis7.GroupIdentifier

I presume this one was just an oversight and needs the annotations scrubbed?

• edu.nps.moves.sql.PduStream
• edu.nps.moves.sql.DatabaseConfiguration

Maybe these two classes can be removed entirely?

The latest version of Macs, Java, and wireless networks together have an annoying problem: you'll be unable to open up any sockets from compiled application. The fix looks like this:

mvn exec:java -Dexec.mainClass=edu.nps.moves.examples.PduSender -Djava.net.preferIPv4Stack=true -Dexec.args="62040 239.1.2.3"

The solution, "-Djava.net.preferIPv4Stack=true", seems to fix Mac laptops only. What makes me wonder if that can cause any problems on Windows, which I don't have handy. If you've got a NetBeans project you can try it out with opening the project->build->run and pasting.

It seems to me that would seem to confirm that "-Djava.net.preferIPv4Stack=true" would make Windows still function.

Currently the library supports being run on JVM's of version 6 or higher. (This is controlled in the pom.xml by maven-compiler-plugin source and target fields).

Leaving support for Java 6 may continue to enable the library to be used in some legacy environments (do we know of any users doing this?)

The downside is we can't use any of the new Java 7 or 8 API's.

I'm thinking we should bump the minimum supported version to at least 7.

I'm in the midst of preparing to cut the next release of open-dis to deploy to Maven Central. I'm fixing up the pom.xml, tweaking the readme a bit, and cleaning up a few misc things that I'll have in the PR. But it got me thinking...

Quite a few very large changes have happened recently, namely dropping hibernate and jaxb support which meant the dismobile and dismobile7 packages were removed. Also, not to mention restructuring the repository when it was moved to github. These things feel like enough to call this release 5.0.

@mcgredonps what do you think?

Fire PDU has two incorrect fields/attributes according to the DIS 1278.1-2012 spec.

• EventId:
  -- According to the DISv7 spec EventId is a Site/Application/Entity field. However in the DISv7 Fire PDU EventId is an EventIdentifier rather than an EntityID like Firing Entity ID, Target Entity ID, and Munition/Expendable Entity ID.

• Descriptor:
  -- Descriptor for DISv7 should be similar to DISv6 Descriptor. However the DISv7 Descriptor is an empty class with default constructor. See:
  edu.nps.moves.disv7.Descriptor.

Hi there,
it would appear there aren't any outstanding Java issues. Is there a record on any? Or a wish list?

I may have effort available to move open-dis-java forward, particularly if there are issues/tasks that would also being benefit to my users.

When useCopies is set to true (as it is by default) createPdu is called twice. During the unmarshal portion of this function the buffer is never saved or reset causing the buffer to start at the incorrect location when createPdu is called again in some cases..

For example, when processing an EntityState Pdu the buffer jumps to 144 after being initially processed causing the copy to be processed starting at 144 rather than 0.

Hi Don, should we close down the SourceForge project soon?

https://sourceforge.net/projects/open-dis/

A fair number of people seem to still be going there and downloading old code and files. I assume they don't know about the GitHub projects and think they are getting the latest content. Is there anything on SourceForge that hasn't been migrated yet to GitHub we'd want to move over first?

Per comments on getUpdatedPositionOrientation(): "This function does not actually perform the computations, it only returns the current state of the entity. The entity state is updated by the specified DR algorithm within the DR class behind the scenes. Updates are created every 1/30 seconds."

It is possible for an observer to query the state while the behind the scenes updates are in progress, returning a partially updated inconsistent state. In my opinion it may not be worthwhile to update this code to make it thread safe; perhaps warning comments should be added regarding this potential issue.

I believe the underlying math that does the updates is correct (both in the DIS_DeadReckoning concrete classes and the unified DeadReckoner implementation) and the thread safety is only an issue when used as described in the quoted comment above.

When marshalling instances of subclasses of edu.nps.moves.dis7.PduSuperclass, the length field is never initialized to anything but zero, so the marshalled PDU advertises length 0.

Further inspection reveals that this is also the case in the DIS 6 PDU types (the setPduLength() method is never called and the pduLength field is never set), but the marshalling code calls getLength(), which delegates to getMarshalledSize() (as opposed to getPduLength(), a getter for the field), calculating the PDU length instead of using the value of the length attribute.

It seems like the marshal call in DIS 7 should also be changed to use getMarshalledSize() or some delegate to it.

In both DIS 6 and 7 PDUs, the length field should probably be removed entirely. This is more robust than having to initialize a field anyway, especially for variable-length PDU types.

I have a fix for this issue, but I was unable to submit pull request. Do I need to be added to a list of users for this project? (It's possible that I'm not using GitHub correctly; I'm using GitHub desktop.)

As a longer term solution, I'd like to replace the Matrix class in the deadreckoning.utils package with Apache Commons Math. However, (1) I don't know whether the project owners are OK with introducing this dependency, and (2) I'm don't know the correct way to include this dependency via Maven. Please advise whether this is a worthwhile update, and if anyone can help with the Maven part.

~Chris

A number of PDU's have a comment in the header that says they are "UNFINISHED". Each of these comments will need to be reviewed to determine what if anything is still required. In some cases, the comment may simply be outdated. In other cases, code changes may be required.

If we do a code search for the UNFINISHED string on GitHub we can see these results:

https://github.com/search?q=repo%3Aopen-dis%2Fopen-dis-java%20UNFINISHED&type=code

When sending PDU's constructed in open-dis python, the length field is zero. During unmarshalling in open-dis-java, an infinite loop is entered because the buffer read pointer is never advanced to the end of the first pdu in the UDP packet.

A pdu with zero in the length field should be detected as invalid and handled accordingly.

Hi, I was wondering if the ElectronicEmissionPdu in dis7 will be fully supported?
I am in need of getting the beam status from the ElectronicEmissionPdu as in section 7.6.2.2.f.5.ix

Could you provide info on a repository where the current release can be found?

I'm trying to send some SignalPdu's and I noticed that the value I put into setDataLength isn't the value that I get back when calling getDataLength.

getDataLength is actually returning data.size() instead of the dataLength member variable.

The spec expects the length to be in bits rather than bytes, so using data.size() ends up being off by a factor of 8.

http://faculty.nps.edu/brutzman/vrtp/mil/navy/nps/disEnumerations/JdbeHtmlFiles/pdu/fb.htm

https://github.com/mcgredonps/open-dis-java/blob/master/src/main/java/edu/nps/moves/dis/SignalPdu.java#L111

There's two obvious ways getDataLength could be changed to fix this, I don't have a strong preference to which fix to go with, but I'm leaning towards A:

A) return dataLength;

B) return data.size() * Byte.SIZE;

I'll send a pull request shortly.

It looks like between DIS 6 and DIS 7, the radio identifier information was stripped out of the Signal PDU. This information includes the Radio Reference ID and the Radio Number that link the Signal PDU to the transmitter and entity.

The information is encapsulated in the RadioIdentifier class so it feels like it should have been included here and was mistakenly left out.

I'd be happy to add it in and submit a PR if I need to.

Release 5.0 doesn't compile, because there is a missing package: edu.nps.moves.disenum. Tested using jdk1.8.0_171

Hi,

I want to ask a question about orientation which is mentioned in "open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/deadreckoning/DIS_DeadReckoning.java" at line no. 153. In fact, standard says that psi is the rotation around z axis whereas at above mentioned line number it is written that "the PSI (rotation about the y-axis)".

Is this deliberate because this will effect calculations?

Regards,
Farhan

Hi, I'am actually implementing IFFDataPdu for the C++ side and trying to figure out how to deal with the record length attributes when marshalling and unmarshalling I discovered also that this part is wrong in the Java side.
https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/dis7/IFFData.java
Specification says that: Record Length—16-bit unsigned integer (6 + K1 + P1) but on the java code i found
dos.writeShort((short) iffData.length);

this should be
dos.writeShort((short)( 6 + iffData.length));

and also no check is performed to assure that the iffData array has been padded in a way to adhere to the 32 bit boundary of the whole record.

At page 281 of the IEEE 1278.1 2012 document it states:

"Record Length. This field shall indicate the record length expressed as the total number of
octets in the record. The value of the Record Length shall be the sum of the sizes of the Record
Type field, the Record Length field, all Record-Specific fields, and any padding required to end
the record on a 32-bit boundary. All Record Length values shall be a multiple of 4. The Record
Length shall be represented by a 16-bit unsigned integer."

So, for me, this means that the whole IFFDataRecord shall be a multiple of 32bit and so due to the fact that recordType and record length are already 6 bytes long the IFFDataRecord shall be of a minimum of 8 bytes long

EDIT:
I have noticed that also the unmarshall is wrong
recordLength = (int) dis.readUnsignedShort(); iffData = new byte[recordLength];
should be something like
recordLength = (int) dis.readUnsignedShort(); iffData = new byte[recordLength-6];
but the specification it's not clear how handle the padding....
What do you think about it? Have you any idea how to find how many bytes are for the data and how many for the padding....
the only things I have found is to differentiate according to the recordtype...each record type has different lengths... most are 16 bit some 128...

Please note, that I'am actually developing the C++ side following this approach

We are using articulated parts parameters:

1. Turret Azimuth
   PRIMARY_TURRET_NUMBER_1 + AZIMUTH
2. Gun Elevation (attached to turret)
   PRIMARY_GUN_NUMBER_1 + ELEVATION

When perform_DR() is called on those parts, the values produced for turret Azimuth seem correct at certain angles, but at other angles, the values flip/flop (even when there is no movement).

In the end, I think that dead-reckoning for those parts should be disabled.

Comment says Gets the Euler Theta value (in radians) from position and Tait-Brayn yaw and roll angles but implementation uses pitch not roll public static double getThetaFromTaitBryanAngles(double lat, double lon, double yaw, double pitch). This is the same for getPsiFromTaitBryanAngles.

In general, I think the previous Dead Reckoning DIS_DR_* implementation should be deprecated in favor of the single DIS_DeadReckoning class I recently added. For example, the previous DIS_DR_* implementations have thread safety issues #41.

Originally posted by @camejia in #42 (comment)

I believe Chris made a typo in that comment and was recommending use of the DeadReckoner class he wrote instead of the DIS_DeadReckoning and DIS_DR_* classes.:

https://github.com/open-dis/open-dis-java/blob/master/src/main/java/edu/nps/moves/deadreckoning/DeadReckoner.java

Another advantage of DeadReckoner is that it has extensive unit tests:

https://github.com/open-dis/open-dis-java/blob/master/src/test/java/edu/nps/moves/deadreckoning/DeadReckonerTest.java

Other issues affecting the classes to be deprecated:

• #80
• #41