Comments (14)
I'm checking it how it is possible to update, but It seems the 1.7 version is quite old so it is not so easy
from orleans.
When the storage library moved to version 2.0 the storage team implemented a new way of using the library. However they kept an implementation that worked with legacy code in Microsoft.WindowsAzure.Storage.Table.DataServices
, a few breaking changes where introduced but on the whole shouldn't be too hard to upgrade the library.
However it might be worth considering updating the whole Orleans codebase to use the newer aspects of the Storage Library, for example the Table Service Layer via TableEntity offers significant performance and latency improvements over the 1.7 lib.
from orleans.
We agree. This is our plan indeed. We even had a change for the APIs prepared but the bigger challenge was the change in errors being returned and their semantics. Need to make sure all errors are handled correctly, in all cases.
from orleans.
A bit off-topic for this thread, why is Microsoft.WindowsAzure.ConfigurationManager
set to version 2.0.0.0 in OrleansAzureUtils?
For what it's worth with my rather small piece of software, I've installed the MSI package and referenced the binaries therein and use the latest Nuget packages for Microsoft.WindowsAzure.Configuration
, Microsoft.WindowsAzure.Storage
etc. with binding redirects and everything looks like working without a hitch (this is how I've ran Orleans starting from the first publicly available bits).
from orleans.
Sounds good. Would you like to contribute it as a pull request?
from orleans.
@gabikliot Heh, I merely meant that I just added the newest libraries, redirects and went off with a fairly small application without much consideration if it really works. At least it works on a emulator and Azure roles as far as I can see. :)
I can give the actual code an actual stab later today. I took a look at AzureTableManager
and some other places quickly and converted almost all of the code in AzureTableManager
(it felt rather straightforward, though I'm not sure about some of the "update rules" on entities). I apologise beforehand already that I may not be able to finish this with the quality I'd be satisfied and try to dish out further testing, quality inspection etc. to other people. Perhaps someone finds the "raw conversion" useful.
I read the following sources and I didn't spot anything breaking error codes at leat:
- http://gauravmantri.com/2012/11/17/storage-client-library-2-0-migrating-table-storage-code/
- http://blogs.msdn.com/b/windowsazurestorage/archive/2012/12/10/updated-known-issues-for-windows-azure-storage-client-library-2-0-for-net-and-windows-runtime.aspx?PageIndex=2
- http://blogs.msdn.com/b/windowsazurestorage/archive/2012/11/06/windows-azure-storage-client-library-2-0-tables-deep-dive.aspx
In addition, I think there are plenty of opportunities to clean up the code and probably make it faster too in the process. One thing to look at is probably also how the Nuget dependencies will be handled. Anyway, I'll try to jot down some notes and make a PR for discussion later today.
from orleans.
I've dived deeper into the Orleans Azure storage implementation and I see that the storage upgrade could be easy if we are staying on the DataServices version (Microsoft.WindowsAzure.Storage.Table.DataServices namespace after the 2.0). Actually it has some improvements, but the whole stuff is legacy and obsolete. It's okay, but I can't see any benefit.
Essentially, more beneficial if we will use the new implementation but it is significant change in the code and interface.
I can see this benefits:
- Performance. The related blog post talk about a 25-75% performance improvements
http://blogs.msdn.com/b/windowsazurestorage/archive/2012/11/06/windows-azure-storage-client-library-2-0-tables-deep-dive.aspx - More simple code because the new storage client implementation has very good retry logic infrastructure and we can remove the current one. Maybe we need to create a custom retry policy, but it is still simple.
- As I see the interface also would be more simple because the TableEntity class contains an ETag and we don't need to handle it in separated parameter
Currently I'm working on the new version, but it is hard to do. But it's worth it because it will be much faster.
@gabikliot Is there any internal integration test to validate the storage change? I mean after the change We should see the error handling and the retry logic are still good.
PR coming soon
from orleans.
@pherbel I updated to the latest Nuget Storage and Configuration packages and refactored the code to use the newer mechanisms. The code compiles and the tests pass, but I don't think it's ready until someone takes it to to an actual spin and checks the code (see my PR notes for more details), but it should give a good direction where to go. If you want to take the code and meddle with it, go ahead. I think the earliest I'm onto this is the next weekend (if I have time).
You are right that using the newer stuff the code would be both faster and (a lot) mor succinct. I'll put some more notes in the next few days. It would seem prudent to plan that refactoring as a separate item.
As a tangential note, is there nothing that stops someone upgrading to .NET 4.5.1 (preferably after this storage update)?
from orleans.
We are all for moving to the latest version, due to all the reasons you mentioned above. The change is indeed far from trivial, but the good thing is it can almost entirely be kept local to one file: AzureTableManager, which abstracts away the Azure Table APIs.
If we have other ideas, beyond moving AzureTableManager to 2.5, how to make things around storage better, lets keep them as separate issues and separate PRs. It will make handling this easier.
from orleans.
I submitted #158 that should complete the initial migration. More refactoring can be done after that.
from orleans.
Fixed via #163.
from orleans.
Awesome! Should we push a NuGet release (or perhaps a prerelease)?
from orleans.
We still want to do another pass, of cleanup, taking some of the additional feedback from @veikkoeeva and others about small perf. improvements, etc... This should not take long now. The biggest challenge for this PR was updating our internal test infrastructure to the new storage version as well. Now that we are done with it all any new fixes should be much easier.
from orleans.
Followed up with #168, which is now merged.
from orleans.
Related Issues (20)
- Bug: Dispose and DisposeAsync not getting called on grains
- Bug: ActivationServices not disposed upon OnActivateAsync throwing
- [Proposal] `IGrainTimer` interface for updatable grain timers
- Could not load type 'Orleans.CodeGeneration.KnownAssemblyAttribute' in Orleans 8.1.0 nuget packages HOT 2
- Cannot cancel an IAsyncEnumerable HOT 3
- GenerateSerializer models don't work with `required` properties HOT 4
- The MembershipTable implementations of Microsoft.Orleans.Clustering.Redis and Microsoft.Orleans.Clustering.AdoNet are inconsistent HOT 2
- OrleansQuery not exists HOT 7
- ObjectDisposedException at Orleans.Runtime.Messaging.MessageCenter.ReceiveMessage
- [bug/feature request] default codec for collections HOT 1
- AWS Kinesis Streaming
- Orleans membership table is not cleared properly from obsolete silos
- `UseUnixSocketConnection` not working
- Ability to skip time to trigger reminders for testing (using TimeProvider)
- Errors on kubernetes hosting with different cluster ids? HOT 1
- How do I place grain according to the silo category?
- Bug: Exclude inaccessible interface methods from codegen HOT 3
- `IGrainReferenceActivatorProvider` exception with interfaces written in F# HOT 1
- [Question] Orlean grain does not allow concurrent read HOT 2
- Typo in silo start log HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from orleans.