Self hosted solution should be better. Right now, the recommended self-hosting solution suggest the pusher file must be placed inside a 'version' specific folder. Other issues make it not ideal. about pusher-js

[adamyeats-zz]

@NathanWalker What are the 'other issues' you mentioned?

pusher-js's Pusher.DependencyLoader loads dependencies dynamically, meaning that if fallbacks are not required, they are not loaded. In most cases, only one GET request will be made for pusher.js. Subsequent requests are dependent on what the browser supports, e.g. if we can't feature detect native JSON support, then we load json2.js, if we need the HTTP fallback then we load sockjs.js.

[NathanWalker]

Other problem is that the recommended solution has an awkward requirement of needing to be put in a folder matching the version in which is being hosted. This should not be the case and the necessary dependencies should be able to be compiled in with other project builds and not be dependent upon any particular folder naming structure.

[NathanWalker]

The configuration should allow specification of where to find any dependencies (at least the fallback .swf file) and that should be able to point to any directory, not dependent on a version number naming scheme.

[adamyeats-zz]

@pl What are your thoughts on this?

[NathanWalker]

I think it's safe to always assume developers (whom you want to use this library) will want to build all of their javascript into 1 single file (to reduce extraneous GET requests) and therefore should provide the ability to configure the library for that very particular case, which is a common one.

[NathanWalker]

I guess at minimum, it would be helpful if the dist folder created the proper version folder when it built, so when user's use bower to install this library, it would contain the proper folder naming scheme required by this current configuration. Right now, if somebody installed this library via bower, they would need to copy the files into a different folder with a name matching the version. Definitely not ideal.

[NathanWalker]

Well another workaround in the current state would be to use the dependencyloader configuration in a wonky way like this:

Pusher.Dependencies = new Pusher.DependencyLoader({
  cdn_http: "http://example.com/bower_components/pusher-js",
  cdn_https: "https://example.com/bower_components/pusher-js",
  version: 'dist',
  suffix: Pusher.dependency_suffix
});

And then I assume everything would work just fine by looking at the src. Of course, that's not very straight-forward but it would work I think.

[NathanWalker]

Just posted a PR to resolve this issue:
#52

[pl]

Hi Nathan,

Files need to be versioned because of caching. For example, if you fetch pusher.js for version 2.1 (without the patch number), you can't load dependencies for just 2.1, because you might get a fresh copy of pusher.js (let's say 2.1.3), but the sockjs.js file might be cached (by CDN, proxies or even the browser) at version 2.1.2. These incompabilities might completely break the library, even for patch versions. You could drop the folder name, but you'd have to handle versioning anyway.

I think the slight inconvenience of handling the folder structure does not justify introducing more complexity into the dependency loader. I don't want to duplicate version information by creating a new directory for each version in the dist directory. Bower is version aware (git tags), so it should allow versioning packages correctly, not putting more burden on the library.

You also don't want to package pusher-js into one file. For majority of connections, only pusher.js is needed. If you packaged everything, you would be serving twice as big file as you need, for probably 60-70% of your users. Additional requests are made only when needed, so in case Flash WebSockets are used or the connection strategy falls back to SockJS. In the future, pusher-js might contain more fallbacks, which would be designed to be fetched dynamically.

[NathanWalker]

Right, pusher.js, is the only file that would be desired to built into a projects single build file. The other's just need to live somewhere not dependent on version number. The problem you are referring to is one that is often resolved quite differently from project to project, therefore, this library should not try to do too much.

If someone is going to do the self-hosted option, they are managing dependencies and versioning themselves and the library should not enforce it's own opinion on that because a different strategy may be being used. Most people going with self-hosted know very well that need to handle versioning and ensure any dependencies are up-to-date and specify their own dependencies which are compatible with the build.

Consider applicationCache, which would handle versioning by downloading the latest 'pusher.js' by making a simple change in the manifest therefore allowing the individual project to ensure an updated pusher is delivered to the client and the dependencies would be specified in a completely different naming convention. Again, this library should not try to do too much for the consumer in this regard.

If anything, place a simple WARNING: in the self-hosted section of the README making mention that you must keep your dependencies up to date, but again, that should be obvious to the developer opting for that strategy.

[NathanWalker]

With the simple PR above, it solves this issue squarely and is all that is needed, and the configuration can be supplied like this:

Pusher.Dependencies = new Pusher.DependencyLoader({
  cdn_http: "http://example.com/bower_components/pusher-js/dist",
  cdn_https: "https://example.com/bower_components/pusher-js/dist",
  suffix: Pusher.dependency_suffix
});

Very simple.

[NathanWalker]

But if the PR is not desired, let's please modify the build to place the dist files in the properly named version folder so bower installs will be seamless and hassle free.

[NathanWalker]

You mention above:

Bower is version aware (git tags), so it should allow versioning packages correctly, not putting more burden on the library.

True but without my PR, you cannot use bower because of that version specific hook you have in the dependency loader.

[pl]

Right, pusher.js, is the only file that would be desired to built into a projects single build file. The other's just need to live somewhere not dependent on version number.

It's important to keep dependencies versioned - if you fetch a wrong version, the library might break. You can fetch pusher.js from any location, because it has a specific version and fetches correct dependencies. The problem is not on the server or in deployment - it's in the nature of the Web - caches are everywhere and you can never be sure what version of each file a client receives, until you start versioning.

I don't want to skip versioning, because it will work in 99.99% of cases, but break from time to time undeterministically. Finding this kind of bugs is a massive pain, I know that from experience.

True but without my PR, you cannot use bower because of that version specific hook you have in the dependency loader.

This is a problem with Bower or its integration, if it does not take into account how caching on the Web works. The dist directory is in the repo only because building distribution files requires a toolchain, which is not always possible to install. I really don't want to duplicate version info in the repository, a git tag is enough, especially when Bower forces me to provide it.

I agree with you that people want to serve dependencies in their own ways, that's why I don't want to enforce any specific algorithm, but we need the default one for our own CDN. You can easily swap the dependency loader for your own implementation without forking the library:

1. inherit from the Pusher.DependencyLoader,
2. overwrite the getRoot method, which is a one-liner,
3. assign an instance of your loader to Pusher.Dependencies.

https://github.com/pusher/pusher-js/blob/ea524df93614261b12d49675859dad0a906e1571/src/dependency_loader.js

It's a much more flexible way to solve this problem, since you can use whatever URL scheme you want and you're not dependent on pusher-js (and me :P).

[NathanWalker]

This is a problem with Bower or its integration, if it does not take into account how caching on the Web works.

That's a confusing conclusion to the problem, which is very clear. Your dependency loader just needs to be flexible in not requiring a version option to be specified. Again, my PR allows for the version to still be specified and README would suggest you specify a version. However, for those needing a more flexible setup, you could omit the version number and provide your own way of versioning.

?

[NathanWalker]

I could inherit your dependency loader but that is quite burdensome given the solution provided in the PR. Not to mention doing as you suggest, only creates more code overhead.

[NathanWalker]

I really don't want to duplicate version info in the repository, a git tag is enough, especially when Bower forces me to provide it.

With the PR, you do not need to do anything else, just please merge it 👍

Otherwise, I will have to depend on our own fork of it which I'd prefer not to do because I would have to keep upstream changes merged in which I will do if I must.

[NathanWalker]

I agree with you that people want to serve dependencies in their own ways..

Awesome.

that's why I don't want to enforce any specific algorithm

But you are, you require a version number be specified when configuring the dependency loader.

but we need the default one for our own CDN.

Perfect, it is setup that way so you are good to go.

You can easily swap the dependency loader for your own implementation without forking the library.

As stated, just not ideal.

[NathanWalker]

and you're not dependent on pusher-js (and me :P).

I want to be dependent on pusher-js and you 👍

You've got a great service and library here. It just needs a small tweak to provide the flexibility that any developer may need :)

[pl]

Your dependency loader just needs to be flexible in not requiring a version option to be specified.

I would rather say that it has to be first correct, then flexible. If you can't prove the correct version of a dependency is fetched, it makes the loader incorrect. Take this scenario as an example:

1. client 1 is on the network A and fetches http://example.com/pusher.js, then http://example.com/sockjs.js, both files have version 2.0.0,
2. you update your website to use 2.1.0,
3. client 2 is on the network B and fetches http://example.com/pusher.js, gets the version 2.1.0,
4. client 2 switches to the network A, has pusher.js cached in version 2.1.0 and has to fetch http://example.com/sockjs.js, but the cache on the network still has version 2.0.0,
5. pusher.js 2.1.0 is not compatible with sockjs.js 2.0.0 and the library breaks for the client 2.

The truth is, if the code is possible to break, someone will eventually do it. README doesn't make things much better, because it requires users to read it. I always try to assume things can break and do everything to prevent it.

Flexibility also doesn't mean that all features need to be built-in, the library can be extensible. This is only one case, there are many more setups and one class can't handle everything. However, there is a generic solution, which requires adding just a few lines of code and will work perfectly with every scheme, even if you wanted to pass an MD5 of the version as a query string param.

I want to be dependent on pusher-js and you 👍

You've got a great service and library here. It just needs a small tweak to provide the flexibility that any developer may need :)

I'm happy to hear that ;) I really want to help you with this one, but the solution has a serious problem and I don't want to bring more support upon our team ;)

[NathanWalker]

Paweł, you're trying to do too much ;)

I stopped at #2.

You're assuming the developer is not managing very carefully their own version of dependencies.

It's not your responsibility to do a diligent developer's job 👍

As a provider of this service and library, you should strive to provide the most versatile options as that always, in the end, provides the best solution.

Look Paweł, you HIGHLY suggest in your README to use your CDN. Period.

You go beyond that, and it's our responsibility.

[NathanWalker]

An over-engineered solution often results in:

1. more code than you need.
2. rigid options to work within varying environments (this is self-evident in the current state of the library given this debate which I am still alive in ;) )
3. answers to problems that may not apply to particular cases which need different answers.

[NathanWalker]

The truth is, if the code is possible to break, someone will eventually do it. README doesn't make things much better, because it requires users to read it. I always try to assume things can break and do everything to prevent it.

The truth is, if someone is going to break something, it's their fault. Not yours.

Especially in this scenario where the case we are talking about is for the self-hosted consumer whom is assuming all risk.

[NathanWalker]

I'm closing this issue and my own PR because I have opted to maintain our own fork of this because this debate has gone in circles and clearly is resulting in inaction, very disappointing.

I will mention that your bower.json file contains no version specification and you will continue to frustrate bower users with this setup.

The debate here has been appreciated, thank you Paweł for your time.

[pl]

Sorry Nathan, but this is a very idealistic point of view. The reality is that when something Pusher-related breaks, we receive support requests, even though in many cases it's not the library's fault. That's completely fine, everyone can have a bad day, misread the docs or make a silly mistake.

I believe libraries should be implemented in a way that minimizes the risk of introducing incorrect behavior. Moreover, client-side libraries should assume as little as possible about the client, as we've been proven numerous times that misbehaving clients exist and always will. These guidelines are there to keep the library reliable and safe to work with.

Having to choose between a correct code and one with a documented flaw, I will always choose the first.

In the case of your patch, the dependency loader would have to trust the client to have consistent or empty caches. This is a very dangerous assumption. I don't want the mainstream code to work this way, so I can't merge your pull request.

I would consider doing something if there was no way to get around it, but there is one. The dependency loader is decoupled and simple enough to implement any URL scheme in a few lines of code.

[NathanWalker]

I can understand that.

Again, you really need to do something about your bower setup still regardless.
For a consumer using your library and installing via bower, doing a bower install pusher will put a pusher directory in the specified bower components folder however the dist folder that comes down contains no version specific folders, therefore using your approach (out of the box) would not work as bower is intended because you would have to further copy files into a version specific folder in a custom build step which again is not using bower as it is intended ( the ability to install dependencies into your project and just include them as is ).

[NathanWalker]

If you have to do additional work to use a library you install via bower, I would consider that not utilizing bower properly to help the consumer of your library.

[pl]

I will think about it. I'm not really sure about changing the dist directory structure:

1. some people might be using current structure already - I don't want to duplicate files,
2. majority of the most popular packages doesn't do it,
3. yet another place when the version number has to be duplicated - I don't like it,
4. bower should take care of versioning, if it wants to call itself a "package manager".

I must say, I don't like the way bower works - it's annoying for both library developers and consumers. So much duplication everywhere.

[NathanWalker]

Paweł, I'm sorry to say, I believe you have missed the entire point of this debate.

bower works perfectly, it supports git tagging as you even acknowledged above:

Bower is version aware (git tags), so it should allow versioning packages correctly, not putting more burden on the library.

Therefore, you do not have to do a thing with the dist directory!@!
Since each tagged version of master would contain it's specific version build of the dist directory, each bower install would get the dist of exactly the git tag that was requested.

The only problem is you require a version be specified in the dependency loader which modifies the directory in which it is to look!! Putting a wrench in the beauty of it all!

When a user wanted to upgrade, they would bump the version in their projects bower.json to pull down the latest files and again, they would be responsible for handling client cache busting of those files, which again, is NOT your responsibility!

[pl]

If you want to continue this discussion, please provide a working solution to the scenario from my previous comment, assuming that:

• you can not modify the dependency loader further (since it's the point of your PR),
• you can't trust browsers that they will obey headers (bugs),
• you can't trust proxies and caches that they will obey headers (bugs, misconfiguration),
• you can't force fetching the JS files on every reload (performance reasons).

Even with a solution that always works, I would have to be convinced that it's significantly better than providing a custom dependency loader.