Comments (12)
Hi,
I'm experiencing the issue with AWS PHP SDK.
AWS SQS sends null when there are no attributes, while ElasticMQ sends d41d8cd98f00b204e9800998ecf8427e (empty string's MD5). (see aws/aws-sdk-php#1315 (comment))
from elasticmq.
Hi @adamw,
I'm experiencing the same issue. I'm no expert in Java/Scala, but might the issue is caused by data conversion? E.g., unicode symbols can be lost if system locale is not unicode and getBytes() is not provided which encoding to use.
So, please consider this patch: https://gist.github.com/kopchik/173e9c3caa99c0a787a90d0e2da85ca6 .
from elasticmq.
Thanks for the report, this looks quite weird, I'll take a look in a while :)
from elasticmq.
Agreed, thanks for building this! Makes testing locally much easier. I actually ran into this same issue and here was how I resolved it.
I was sending messages using the Java AWS SDK. My messages looked kind of weird when sent as a String because I was actually just sending a byte[] wrapped as a String. It turns out that there are some invalid characters that SQS does not accept (perhaps the mdash in the email subject in this example). I'm not sure exactly what is happening, but I think the MD5 calculation is different because one message body is sanitized and the other version is not.
I fixed it by just Url Safe base64 encoding first.
from elasticmq.
Unfortunately I can't reproduce this locally. There are indeed forbidden characters in SQS, but such messages should be rejected and are rejected by both SQS and ElasticMQ. I tried the message body from above but works fine with the latest version.
Please reopen if anybody encounters this problem again.
from elasticmq.
I'm getting this problem now.
I had some integration tests which were all passing. I had a requirement to add a couple of new attributes to the message.
I ran the tests. They passed. I added the 2 new attributes. Then I got:
Attribute MD5 hash mismatch
I then removed the 2 attributes and goot:
Attribute MD5 hash mismatch for message id 760349d3-0d7f-4593-bfa0-1a15cb0c6ca3
The queue is now fucked. I literally added 2 attributes. Then removed them. Now I can't do anything.
from elasticmq.
When adding the message attributes, we have mixed casing on the names. For example:
As you can see here, ftpuser
and ftpaddress
are lowercase. While the other two are uppercase.
If I change the two values to be FtpUser
and FtpAddress
my error goes away.
It's like there is inconsistency when doing the the MD5 hash but I can only find the validation one, I cannot find the send request one to compare what they are doing.
from elasticmq.
Can confirm, you have a casing bug in your SDK.
If the attribute names are all uppercase, or all lowercase. There's 0 issues.
If you have some upper some lower it fails 100% of the time.
You can read not but write, or not write, depending on the order you attempt to do this stuff in.
from elasticmq.
So it's a bug somewhere in ElasticMQ you say? :)
from elasticmq.
@adamw I'm not sure if it's related to ElasticMQ, we are using SQS. I didn't realise this was logged under ElasticMQ because this was like the only link that popped up with the error message.
So it's a bug with SQS, not system.
from elasticmq.
Ah ok, thanks for the clarification :)
from elasticmq.
@kopchik Ah! nice catch, could be :) Could you maybe convert the gist into a PR? that way you'll be properly included in the history :)
from elasticmq.
Related Issues (20)
- Actor System does not shut down when using `SQSRestServerBuilder#stopAndWait` - resulting in main process hanging indefinitely HOT 2
- prebuilt arm builds for softwaremill/elasticmq-native is not working HOT 1
- Stale Documentation for DeadLetterQueue creation HOT 2
- Publishing messages with delay does not work as expected HOT 1
- Could not find UI files error on 1.4.5-RC1 docker image HOT 2
- Fifo delay queue is delivering messages immediately instead of delaying them HOT 2
- Migrate from akka to pekko? HOT 6
- Automatic creation of queue does not work (v1.4.6) HOT 1
- IllegalArgumentException: Couldn't find header X-Amz-Target HOT 6
- x-amz-json-1.0 not properly serializing error responses HOT 5
- AWS JSON Protocol - MessageAttributes key has appeared? HOT 3
- JSON protocol support HOT 1
- DeadLetterQueueSourceArn missing while receiving message from DLQ HOT 1
- Push images to ECR HOT 2
- Trigger Events Based on Messages in Queue HOT 4
- Exception when running routes spray.json.DeserializationException HOT 3
- "MD5 returned by SQS does not match the calculation on the original request" when using AWS JSON protocol and message attributes with custom data types HOT 1
- java.lang.IllegalArgumentException: Couldn't find header X-Amz-Target
- Cannot create Queue with AWS SDK. HOT 6
- 404s after updating AWS SDKs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from elasticmq.