Clarify docs using suggestions from WF forums about letsencrypt-webfaction

[will-in-wi]

I'm going to copy parts of the comments inline and record how I addressed them here.

(Q0) Would using https://certbot.eff.org make any of this easier?

Possibly. My understanding of Certbot is that it only works in a situation where you have a web facing server with root/administrator access. In the case of Webfaction, the latter constraint is not the case. I've added a section to the Readme entitled "Why not Certbot" to explain this.

(Q1) What's cron? (I have a vague idea, but need info on how to run it.)
(Q2) How do I access it?
(Q3) What are the commands?

Cron is a huge topic and I assume a basic knowledge. However to make this clearer (I hope!), I added a section to the Readme called "Prerequisite topics" which links to the WebFaction documentation for cron.

You can install LetsEncrypt Webfaction using the system Ruby or using RBEnv.
(Q4) Why would I want to use the more complex method to install?

I tried to add further clarification to the RBenv section as to why you would use it. I personally use RBenv to manage my Ruby installations since I develop and run a set of Rails applications. I just wanted to describe this method since it was (a) the one I personally use and (b) because it avoids a "dual stack" approach to Ruby.

System Ruby:
Run the following command to install:
(Q5) Where do I run it? I have several sites. Do I run this in the same directory as .bash_profile? (What's this directory called in shared-server parlance?)
(Q6) How do I access the server to run it?

Good point! In the new SSH section, I clarified that all of the commands are run via an SSH client. I also linked to the WebFaction documentation which describes how to get SSH set up.

GEM_HOME...
(Q7) Is $HOME what I literally type, or is this something I need to substitute? (This isn't clear throughout. It would be good to state at the top that "x" should be substituted by "y" in all the commands.)

I've added a note at the top of the Readme that when substitution needs to occur, the substituted section will be surrounded by square brackets. I've also gone through the Readme and made all of this consistent.

(Q8) What does this command do? (Remember that running commands from advice you saw on a page on the internet is generally not recommended for the average user, so some pointers that reassure what the commands do would be useful.)

Good point. I've tried to add clarification to all commands.

(Q9) If I'm installing something, will I need to make sure in future that it is up to date with fixes etc? (This whole area is not clear to me: does Webfaction update my server's software, or is it my responsibility? And for stuff that I install like this, who's responsible?)

Good point! There are two issues here: 1) server software and 2) applications like LetsEncrypt Webfaction which are installed by the user.

Regarding 1, the simple answer is that WebFaction will maintain server software. However, this is slightly more complex than it seems at first blush. If you install a webapp through a WebFaction installer (like Rails or Wordpress), the webapp will not be automatically updated. Additionally, if you use Rails, you will have a local Nginx server which is being proxied to by the frontend Nginx servers. The frontend Nginx server is updated by WebFaction. The application specific Nginx server is not updated by WebFaction. When in doubt, I suggest opening a support ticket and asking. The support teams seems more than happy to answer such questions.

Regarding 2, you are responsible for upgrading LetsEncrypt Webfaction. I've added an Upgrading section to the Readme which clarifies this and describes how to do so (and found and fixed a bug in the process).

Then run source $HOME/.bash_profile
(Q10) Is this a command? That I type as-is?

Yes. I've clarified.

Now, you can run letsencrypt_webfaction from the shell.
(Q11) Do I run this in the same directory as .bash_profile?

You don't have to. .bash_profile is user-wide. I've added a note to clarify.

(Q12) What does this command do?

Gets certificates! 😄 I've clarified.

Usage
(Q13) Usage for what? To generate a certificate?

Yes, I've clarified this.

(Q14) What is the account_email? Does it have to be the email that I have associated with Webfaction, or can it be anything? (There's more on this below in the documentation, but it should be here.)

I've briefly clarified at this point in the docs.

(Q15) Can the domains be anything? So if I host example1.com, www.example1.com, example1.xyz, www.example1.xyz, and example2.com on this webfaction account can I specify all of them in one command? Will this create 5 certs, or 1 cert that's used for all?

I've adjusted the language to clarify that this generates one certificate for multiple domains. I had someone else email me who was also confused on this point. Thanks for reminding me to clarify.

--public ~/webapps/myapp/public_html/
(Q16) Do I need to substitute something for "myapp"? What?

In the square bracketing of all the things as noted above, I think this has been clarified.

(Q17) I have several webfaction apps for the various domains I have. Do I therefore need to run the letsencrypt_webfaction command for each app?

Yes. I clarified this in the fix for the previous question.

(Q18) Where will the certificate be generated? (There's more on this below in the documentation, but it should be here.)

Added a note to this effect.

(Q19) What do I need to send to Webfaction to get it "activated" (not sure the correct term)? (Again, there's more on this below in the documentation, but it should be here.)

Also added here. 😄

Cron usage
(Q20) Where do I put this task?

In the crontab. The WebFaction documentation I've now linked to at the top of the Readme covers this.

(Q21) If the certs expire every 90 days, why do you renew every 60 or so? Is it OK to renew before expiry?

To give you a month just in case this fails. Yes, this behavior is well within the rate limits (unless you are issuing a ton of certificates.

(Q22) Do I need to do anything else?

I don't think so…

Detailed examples
I won't cover most of this section, as it's for more advanced users anyway, but regarding:
A config file needs to be in YAML format and have a subset of the keys in config.defaults.yml. If you use a config file, you pass the --config ./myconfig.yml parameter.
(Q23) Pass to which command? letsencrypt_webfaction?

Yes. I've clarified.

Operation
When the code runs...
(Q24) Which code? From which command? The command in the "Usage" section? It would be better if this info were placed above, where the relevant command is listed.

"Code" is a confusing term, so I switched it out. I also clarified the section title.

If you see messages containing SyntaxErrors, you are most likely using an old version of Ruby. This utility requires Ruby 2.1+

I deleted this section. It originates from an earlier installation method which could possibly end up using an older Ruby version. It is not likely to be relevant anymore.

(Q25) How do I know what version I have? Am I responsible for updating it? Isn't this standard?

Assuming the version referenced is the Ruby version, if you use the system Ruby installation method you can find out what Ruby version is used by running ruby2.2 --install. WebFaction support automatically updates this for you.

Public folders

For this utility to work, it is assumed that there is a folder which is directly served at http://yourdomain/ into which the ACME verification files can be placed
Not too clear what this means:

I've struggled to articulate this better. If you have suggestions, I'd love to hear them.

(Q26) If I have a website that an internet user can see when they type "http://example.com" I'm OK?

Yes.

(Q27) If I have used the standard WordPress install on Webfaction, am I OK?

Assuming you have a Website to connect the Domain and Application, yes.

(Q28) What if I have set up the WordPress installation to point to http://example.com/wordpress (I haven't; phew!)

That gets trickier. You need to point at the folder that serves http://example.com/. Let's Encrypt has to authenticate against http://example.com/.well-known/ which doesn't work with a subfolder.

(Q29) What about a standard Webfaction static site?

This should also work as long as it serves the root of the domain as described above. The verification process is a simple as placing a file in a folder and telling Let's Encrypt to look for it.

(Q30) Is this is a pre-req for the whole thing to work? If so, then it should be placed at the top of the doc, not the bottom.

Excellent point, I've moved it up.

Custom email configuration
Particularly in the case of Gmail, you may need to override the default usage of Sendmail and use SMTP.
(Q31) Where do I put this info?

I've tried to clarify this.

(Q32) What if it's a webfaction-defined address on one of the domains I'm creating the certificate for (which I guess will be a pretty common situation)? Will I need to do anything in that case?

No. It should work just fine using the default settings.

See the wiki
(Q33) Which wiki?

The GitHub wiki. Tab at the top of the page. https://github.com/will-in-wi/letsencrypt-webfaction/wiki

I clarified this slightly.

Development
No idea what most of this section means.
(Q34) Does it matter?

Not really. I added a line to clarify that this information is only really useful if you are going to develop new code.

But this bit looks important and should be further up the document:

To test certificate issuance, consider using the LetsEncrypt staging server. This doesn't have the 5 certs per domain every 7 days rate limit. You can add the --endpoint https://acme-staging.api.letsencrypt.org/ parameter to do so.

Very good point. I've moved this up.

(Q35) Where? To the letsencrypt_webfaction command? Or in a (which) config file?

To the command. I've clarified.

Other
(Q36) Once the cert is installed on my standard Webfaction Wordpress site, what do I need to do to make it redirect to https instead of http? (I guess this is something that should be in the webfaction documentation, not here, but a pointer to it would be useful.)

I've added a link to this: https://docs.webfaction.com/software/static.html#static-redirecting-from-http-to-https

[ElectricFeet]

This is looking really good. Thanks for being so accepting of the critique. While "Good critiques are difficult to come by", good responses to good critiques are even rarer :)

Thanks for re-working the doc so quickly and comprehensively. Inevitably, this raises a few more questions!

• I'm still confused by "(or your ACME provider)" which I simply deleted :-) but then saw a reference to ACME verification files further down again :-( You ask if I can articulate better, but I don't know what it means, so cannot:

I (now) know that there are things called "ACME verification files" and I don't know what these are. But I can accept that they are used in the process, without me needing to know what they are. I guess the bit I really don't understand is "it is assumed that there is a folder which is directly served at http://yourdomain/ into which [they] can be placed." Not understanding servers, why would there ever NOT be such a folder? Maybe a counter-example would help? That is, give a specific circumstance on Webfaction where such a folder would not exist?

• Re "/public_html/": If I installed via a standard WordPress one(ish)-click installation and didn't select a subdirectory (so my app is "myapp" and webapps/myapp contains my index.php), do I need /public_html/? If not, I think that this might be better clarified by referring to where the index.php file is (as you do further down), as this is a great way for newbies to figure stuff out by looking at the directory structure in an FTP client. My understanding as a server newbie is that I have my domain that's served (I think) from "myapp" folder and don't understand when I would have a /public_html/ involved.
• I redid the square bracket/angle bracket thing and tried to make it more consistent with standard usage for
  parameters and options (see, for example, more on this here: http://pubs.opengroup.org/onlinepubs/009696899/basedefs/xbd_chap12.html). I may have failed here and there,
• I also updated the doc and made quite a few assumptions, based on my growing knowledge and your clarifications. I'm pretty sure your clarifications are fine :-) but my growing knowledge is still a dangerous thing! (I think I know enough now to confidently get myself into trouble, but not yet know how to get out of it again.)

So the pull request needs checking carefully, to make sure I haven't introduced errors

Other notable changes:

• I moved the upgrading further down, as it's something you'd do well after the first run
• Several other sections are moved and Git unfortunately lists these as a big deletion. Some manual comparisons will be necessary. Sorry!
• I tried to standardize some of the "myapp" / "you" / "yourdomain" / "my_public_html", settling on the use of "you" (the reader).
• I added the "[.../public_html]/" to the cron job commands too, as otherwise they are inconsistent. I presume this is correct.

One extra question, if I may, from the original post: Why did I get this error when I first used SSH?

The authenticity of host 'my_server_name.webfaction.com my_ip_address' can't be established.
ECDSA key fingerprint is SHA256: ... (hash)...
Are you sure you want to continue connecting (yes/no)?

Is it worth mentioning in the doc? (I realise that this should really be better addressed by WebFaction's documentation, but it's an error people may encounter with LetsEncrypt WebFaction, so could be addressed here too.)

Looking forward to trying it out once I've understood the final few niggles!

Apologies if the Git stuff is messy. This is the first time (another learning curve!) I've tried to do a proper pull request. First I did a branch and tried to pull it, but I was pulling into your master, so it didn't work. Then I forked it and tried to pull, but I'm not sure if I pulled into the master or my own. It still says that some checks haven't finished yet... ho hum.

[will-in-wi]

This is wonderful! I made a couple comments regarding mostly markdown syntax.

To address your comments:

I'm still confused by "(or your ACME provider)" which I simply deleted :-) but then saw a reference to ACME verification files further down again :-( You ask if I can articulate better, but I don't know what it means, so cannot:
I (now) know that there are things called "ACME verification files" and I don't know what these are. But I can accept that they are used in the process, without me needing to know what they are. I guess the bit I really don't understand is "it is assumed that there is a folder which is directly served at http://yourdomain/ into which [they] can be placed." Not understanding servers, why would there ever NOT be such a folder? Maybe a counter-example would help? That is, give a specific circumstance on Webfaction where such a folder would not exist?

The case I've run into where such a folder doesn't exist are some Node.js or Python applications. In some of those cases, all the js/css/images are served through the application itself, so arbitrary files don't really work. One ticket regarding this exact issue is here: #24. There are some workarounds.

ACME is the name of the protocol specification which Let's Encrypt uses to validate and issue certificates. Theoretically, other companies could implement the specification and then work with all of the ACME clients (including Certbot). The idea of ACME might be slightly complicated, but it is technically correct… We could probably just stop referring to ACME until the day when someone might actually want to use this app with some other ACME compliant Certificate Authority.

Re "/public_html/": If I installed via a standard WordPress one(ish)-click installation and didn't select a subdirectory (so my app is "myapp" and webapps/myapp contains my index.php), do I need /public_html/? If not, I think that this might be better clarified by referring to where the index.php file is (as you do further down), as this is a great way for newbies to figure stuff out by looking at the directory structure in an FTP client. My understanding as a server newbie is that I have my domain that's served (I think) from "myapp" folder and don't understand when I would have a /public_html/ involved.

You are correct that in a PHP situation it would be wherever the root index.php resides. I personally do so little PHP these days that I forget that this is probably the majority of my audience. It might be worth clarifying this for the common case.

I've used the public_html/ mechanism commonly for custom PHP applications I've built. Because PHP applications are just a custom way of serving a file, every file under the htdoc root (where your index.php file is) will also be served by Apache unless you manually blacklist files from being served. You could imagine a case where you configure a database.php file in the same location as index.php. But the developer misconfigured this so that if you went to database.php directly in the browser, it might show the database credentials. A common way around this is to only store the index.php and static assets in public_html/ (or some other named directory) and then put the rest of the PHP in a folder above it.

I redid the square bracket/angle bracket thing and tried to make it more consistent with standard usage for parameters and options (see, for example, more on this here: http://pubs.opengroup.org/onlinepubs/009696899/basedefs/xbd_chap12.html). I may have failed here and there.

This is splendid. I think it is much more readable.

I also updated the doc and made quite a few assumptions, based on my growing knowledge and your clarifications. I'm pretty sure your clarifications are fine :-) but my growing knowledge is still a dangerous thing! (I think I know enough now to confidently get myself into trouble, but not yet know how to get out of it again.)

So the pull request needs checking carefully, to make sure I haven't introduced errors

I've read through it once and I think your intuition is spot on. I'll go through it again more carefully as we progress.

Other notable changes:

I moved the upgrading further down, as it's something you'd do well after the first run
Several other sections are moved and Git unfortunately lists these as a big deletion. Some manual comparisons will be necessary. Sorry!

Occupational hazard of working with diffs. 😄

I tried to standardize some of the "myapp" / "you" / "yourdomain" / "my_public_html", settling on the use of "you" (the reader).

This is really useful.

I added the "[.../public_html]/" to the cron job commands too, as otherwise they are inconsistent. I presume this is correct.

Makes sense.

One extra question, if I may, from the original post: Why did I get this error when I first used SSH?

The authenticity of host 'my_server_name.webfaction.com my_ip_address' can't be established.
ECDSA key fingerprint is SHA256: ... (hash)...
Are you sure you want to continue connecting (yes/no)?

Is it worth mentioning in the doc? (I realise that this should really be better addressed by WebFaction's documentation, but it's an error people may encounter with LetsEncrypt WebFaction, so could be addressed here too.)

Good question… This is a standard SSH message which anyone who has connected to 2-3 servers will immediately recognize. It means that the first time you connect to the server, your client doesn't have any way of knowing whether the server key it is handing you actually belongs to the server in question. So it shows this message. The subsequent connections will look at the known_hosts file in your ssh configuration and compare the stored domain/key with what is being served by the server. The security benefit of this is that if someone were to redirect you to a malicious server, they wouldn't have the same key, and thus your SSH client would block you from connecting. Since it can't do that check the first time, it warns you.

Common issue, and I'm tempted to chalk it up as basic SSH knowledge which I'm not trying to document. However if you think that it would be a roadblock, it is probably worth briefly mentioning.

Looking forward to trying it out once I've understood the final few niggles!

I will consider this a roaring success when I hear that you have a site working with this. 😄

Apologies if the Git stuff is messy. This is the first time (another learning curve!) I've tried to do a proper pull request. First I did a branch and tried to pull it, but I was pulling into your master, so it didn't work. Then I forked it and tried to pull, but I'm not sure if I pulled into the master or my own. It still says that some checks haven't finished yet... ho hum.

Always a first time. Thanks for persevering! You don't have enough access to my repository to break anything, so don't worry about it. 😄 For future reference, I believe that if you end your commit message with [skip ci] the test suite won't run when you push. In the case where all you are changing is the readme, the automated test suite won't be checking anything you changed, so it is superfluous. However, it doesn't hurt either.

Thanks again! I'll wait to hear back from you if you need more clarification or if you've made more fixes you want reviewed. My personal pet peeve with pull requests is when I feel like I've put work into them and then they get ignored. So I want to make sure that you feel like you know what, if anything, is needed before merge. As soon as it is ready, I'll merge it.

[ElectricFeet]

Thanks! This is all getting very much clearer.

Do I need to do anything from my side now? I'd be happy to work in a few of the above explanations if you want me to. Or are you working on it? I'm not sure if there's a way in Git to see if you're working on it--I guess there would be a branch if so(?)

[will-in-wi]

I'm not working on anything at the moment. If you would like to take a whack at the remaining items, feel free.

Here is the list of things I can think of as needing to be done:

• Line 110 in your PR needs to be reworked somewhat to avoid the left/right scrolling.
• Any changes you would like to make based on what I described above.

If there are any particular sections you would like me to take a crack at writing, please ask!

Thanks!

[ElectricFeet]

OK. Will do!

[will-in-wi]

If I am working on something, I'll say that I am so that we don't both end up doing the same thing.

[ElectricFeet]

OK. I'm done. I've saved to my branch (fork? still not sure of the terms). Not sure if I have to do anything more to enable you to merge it. Let me know.

I did the following:

Fixed all the formatting of the parameters. Seems to work OK.

Explained that the parameters listed are only the basic ones, and more detail is in the config file.

Moved (and tweaked) the --support_email stuff from the dev section to the "Testing" section (where it's needed first) and then referred back to it in the dev section.

Gave a full command example for testing scenario --- I'll be needing it this week! :-)

Clarified need to create the folder for the ACME verification files in some Node.js/Python scenarios, based on your explanations.

Added link for YAML format.

Thanks for all your help. Tomorrow or Monday I'll be giving it all a go ... :-)

[will-in-wi]

Merged! I'll continue to iterate on this, but I want to get your big set of improvements in first. Thanks for all of your help, and please feel free to continue to contribute improvements as you think of them.