pax_global_header00006660000000000000000000000064141445437730014526gustar00rootroot0000000000000052 comment=0db445ac897cc83df0aaf1bbfc94bd7370dd69bb golang-github-caddyserver-certmagic-0.15.2/000077500000000000000000000000001414454377300206075ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.15.2/.github/000077500000000000000000000000001414454377300221475ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.15.2/.github/CONTRIBUTING.md000066400000000000000000000232061414454377300244030ustar00rootroot00000000000000Contributing to CertMagic ========================= ## Common Tasks - [Contributing code](#contributing-code) - [Reporting a bug](#reporting-bugs) - [Suggesting an enhancement or a new feature](#suggesting-features) - [Improving documentation](#improving-documentation) Other menu items: - [Values](#values) - [Thank You](#thank-you) ### Contributing code You can have a direct impact on the project by helping with its code. To contribute code to CertMagic, open a [pull request](https://github.com/caddyserver/certmagic/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with CertMagic's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/caddyserver/certmagic). We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable. Here are some of the expectations we have of contributors: - If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, comment on the existing issue to claim it. - **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184) - [**Don't "push" your pull requests.**](https://www.igvita.com/2011/12/19/dont-push-your-pull-requests/) Basically, work with—not against—the maintainers -- theirs is not a glorious job. In fact, consider becoming a CertMagic maintainer yourself! - **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other. - **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass. - **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling. - **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft ` then `git commit -s`. - **Maintain your contributions.** Please help maintain your change after it is merged. - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious, and comments should not state the obvious. We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base! ### HOW TO MAKE A PULL REQUEST TO CERTMAGIC Contributing to Go projects on GitHub is fun and easy. We recommend the following workflow: 1. [Fork this repo](https://github.com/caddyserver/certmagic). This makes a copy of the code you can write to. 2. If you don't already have this repo (caddyserver/certmagic.git) repo on your computer, get it with `go get github.com/caddyserver/certmagic`. 3. Tell git that it can push the caddyserver/certmagic.git repo to your fork by adding a remote: `git remote add myfork https://github.com/you/certmagic.git` 4. Make your changes in the caddyserver/certmagic.git repo on your computer. 5. Push your changes to your fork: `git push myfork` 6. [Create a pull request](https://github.com/caddyserver/certmagic/pull/new/master) to merge your changes into caddyserver/certmagic @ master. (Click "compare across forks" and change the head fork.) This workflow is nice because you don't have to change import paths. You can get fancier by using different branches if you want. ### Reporting bugs Like every software, CertMagic has its flaws. If you find one, [search the issues](https://github.com/caddyserver/certmagic/issues) to see if it has already been reported. If not, [open a new issue](https://github.com/caddyserver/certmagic/issues/new) and describe the bug clearly. **You can help stop bugs in their tracks!** Speed up the patching process by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too. Please follow the issue template so we have all the needed information. Unredacted—yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. The burden is on you to convince us that it is actually a bug in CertMagic. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you! Failure to fill out the issue template will probably result in the issue being closed. We suggest reading [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html). Please be kind. :smile: Remember that CertMagic comes at no cost to you, and you're getting free support when we fix your issues. If we helped you, please consider helping someone else! ### Suggesting features First, [search to see if your feature has already been requested](https://github.com/caddyserver/certmagic/issues). If it has, you can add a :+1: reaction to vote for it. If your feature idea is new, open an issue to request the feature. You don't have to follow the bug template for feature requests. Please describe your idea thoroughly so that we know how to implement it! Really vague requests may not be helpful or actionable and without clarification will have to be closed. **Please do not "bump" issues with comments that ask if there are any updates.** While we really do value your requests and implement many of them, not all features are a good fit for CertMagic. If a feature is not in the best interest of the CertMagic project or its users in general, we may politely decline to implement it. ## Collaborator Instructions Collabators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help.The expectations we have of collaborators are: - **Help review pull requests.** Be meticulous, but also kind. We love our contributors, but we critique the contribution to make it better. Multiple, thorough reviews make for the best contributions! Here are some questions to consider: - Can the change be made more elegant? - Is this a maintenance burden? - What assumptions does the code make? - Is it well-tested? - Is the change a good fit for the project? - Does it actually fix the problem or is it creating a special case instead? - Does the change incur any new dependencies? (Avoid these!) - **Answer issues.** If every collaborator helped out with issues, we could count the number of open issues on two hands. This means getting involved in the discussion, investigating the code, and yes, debugging it. It's fun. Really! :smile: Please, please help with open issues. Granted, some issues need to be done before others. And of course some are larger than others: you don't have to do it all yourself. Work with other collaborators as a team! - **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too). - **Prefer squashed commits over a messy merge.** If there are many little commits, please [squash the commits](https://stackoverflow.com/a/11732910/1048862) so we don't clutter the commit history. - **Don't accept new dependencies lightly.** Dependencies can make the world crash and burn, but they are sometimes necessary. Choose carefully. Extremely small dependencies (a few lines of code) can be inlined. The rest may not be needed. - **Make sure tests test the actual thing.** Double-check that the tests fail without the change, and pass with it. It's important that they assert what they're purported to assert. - **Recommended reading** - [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments) for an idea of what we look for in good, clean Go code - [Linus Torvalds describes a good commit message](https://gist.github.com/matthewhudson/1475276) - [Best Practices for Maintainers](https://opensource.guide/best-practices/) - [Shrinking Code Review](https://alexgaynor.net/2015/dec/29/shrinking-code-review/) ## Values - A person is always more important than code. People don't like being handled "efficiently". But we can still process issues and pull requests efficiently while being kind, patient, and considerate. - The ends justify the means, if the means are good. A good tree won't produce bad fruit. But if we cut corners or are hasty in our process, the end result will not be good. ## Thank you Thanks for your help! CertMagic would not be what it is today without your contributions.golang-github-caddyserver-certmagic-0.15.2/.github/FUNDING.yml000066400000000000000000000013171414454377300237660ustar00rootroot00000000000000# These are supported funding model platforms github: [mholt] # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2] patreon: # Replace with a single Patreon username open_collective: # Replace with a single Open Collective username ko_fi: # Replace with a single Ko-fi username tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry liberapay: # Replace with a single Liberapay username issuehunt: # Replace with a single IssueHunt username otechie: # Replace with a single Otechie username custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] golang-github-caddyserver-certmagic-0.15.2/.github/ISSUE_TEMPLATE/000077500000000000000000000000001414454377300243325ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.15.2/.github/ISSUE_TEMPLATE/bug_report.md000066400000000000000000000024451414454377300270310ustar00rootroot00000000000000--- name: Bug report about: For behaviors which violate documentation or cause incorrect results title: '' labels: '' assignees: '' --- ## What version of the package are you using? ## What are you trying to do? ## What steps did you take? ## What did you expect to happen, and what actually happened instead? ## How do you think this should be fixed? ## Please link to any related issues, pull requests, and/or discussion ## Bonus: What do you use CertMagic for, and do you find it useful? golang-github-caddyserver-certmagic-0.15.2/.github/ISSUE_TEMPLATE/feature_request.md000066400000000000000000000020011414454377300300500ustar00rootroot00000000000000--- name: Feature request about: Suggest an idea for this project title: '' labels: feature request assignees: '' --- ## What would you like to have changed? ## Why is this feature a useful, necessary, and/or important addition to this project? ## What alternatives are there, or what are you doing in the meantime to work around the lack of this feature? ## Please link to any relevant issues, pull requests, or other discussions. golang-github-caddyserver-certmagic-0.15.2/.github/ISSUE_TEMPLATE/question.md000066400000000000000000000015231414454377300265240ustar00rootroot00000000000000--- name: Question about: For help or questions about this package title: '' labels: question assignees: '' --- ## What is your question? ## What have you already tried? ## Include any other information or discussion. ## Bonus: What do you use this package for, and does it help you? golang-github-caddyserver-certmagic-0.15.2/.github/workflows/000077500000000000000000000000001414454377300242045ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.15.2/.github/workflows/ci.yml000066400000000000000000000026411414454377300253250ustar00rootroot00000000000000# Used as inspiration: https://github.com/mvdan/github-actions-golang name: Tests on: push: branches: - master pull_request: branches: - master jobs: test: strategy: matrix: os: [ ubuntu-latest, macos-latest, windows-latest ] go: [ '1.14', '1.15' ] runs-on: ${{ matrix.os }} steps: - name: Install Go uses: actions/setup-go@v2 with: go-version: ${{ matrix.go }} - name: Checkout code uses: actions/checkout@v2 - name: Print Go version and environment id: vars run: | printf "Using go at: $(which go)\n" printf "Go version: $(go version)\n" printf "\n\nGo environment:\n\n" go env printf "\n\nSystem environment:\n\n" env # Calculate the short SHA1 hash of the git commit echo "::set-output name=short_sha::$(git rev-parse --short HEAD)" echo "::set-output name=go_cache::$(go env GOCACHE)" - name: Cache the build cache uses: actions/cache@v2 with: path: ${{ steps.vars.outputs.go_cache }} key: ${{ runner.os }}-${{ matrix.go }}-go-ci-${{ hashFiles('**/go.sum') }} restore-keys: | ${{ runner.os }}-${{ matrix.go }}-go-ci - name: Get dependencies run: | go get -v -t -d ./... - name: Run tests run: | go test -v -coverprofile="cover-profile.out" -short -race ./...golang-github-caddyserver-certmagic-0.15.2/.gitignore000066400000000000000000000000141414454377300225720ustar00rootroot00000000000000_gitignore/ golang-github-caddyserver-certmagic-0.15.2/LICENSE.txt000066400000000000000000000261351414454377300224410ustar00rootroot00000000000000 Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright {yyyy} {name of copyright owner} Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. golang-github-caddyserver-certmagic-0.15.2/README.md000066400000000000000000000676621414454377300221070ustar00rootroot00000000000000

CertMagic

Easy and Powerful TLS Automation

The same library used by the Caddy Web Server

Caddy's [automagic TLS features](https://caddyserver.com/docs/automatic-https)—now for your own Go programs—in one powerful and easy-to-use library! CertMagic is the most mature, robust, and powerful ACME client integration for Go... and perhaps ever. With CertMagic, you can add one line to your Go application to serve securely over TLS, without ever having to touch certificates. Instead of: ```go // plaintext HTTP, gross 🤢 http.ListenAndServe(":80", mux) ``` Use CertMagic: ```go // encrypted HTTPS with HTTP->HTTPS redirects - yay! 🔒😍 certmagic.HTTPS([]string{"example.com"}, mux) ``` That line of code will serve your HTTP router `mux` over HTTPS, complete with HTTP->HTTPS redirects. It obtains and renews the TLS certificates. It staples OCSP responses for greater privacy and security. As long as your domain name points to your server, CertMagic will keep its connections secure. Compared to other ACME client libraries for Go, only CertMagic supports the full suite of ACME features, and no other library matches CertMagic's maturity and reliability. CertMagic - Automatic HTTPS using Let's Encrypt =============================================== ## Menu - [Features](#features) - [Requirements](#requirements) - [Installation](#installation) - [Usage](#usage) - [Package Overview](#package-overview) - [Certificate authority](#certificate-authority) - [The `Config` type](#the-config-type) - [Defaults](#defaults) - [Providing an email address](#providing-an-email-address) - [Rate limiting](#rate-limiting) - [Development and testing](#development-and-testing) - [Examples](#examples) - [Serving HTTP handlers with HTTPS](#serving-http-handlers-with-https) - [Starting a TLS listener](#starting-a-tls-listener) - [Getting a tls.Config](#getting-a-tlsconfig) - [Advanced use](#advanced-use) - [Wildcard Certificates](#wildcard-certificates) - [Behind a load balancer (or in a cluster)](#behind-a-load-balancer-or-in-a-cluster) - [The ACME Challenges](#the-acme-challenges) - [HTTP Challenge](#http-challenge) - [TLS-ALPN Challenge](#tls-alpn-challenge) - [DNS Challenge](#dns-challenge) - [On-Demand TLS](#on-demand-tls) - [Storage](#storage) - [Cache](#cache) - [Contributing](#contributing) - [Project History](#project-history) - [Credits and License](#credits-and-license) ## Features - Fully automated certificate management including issuance and renewal - One-liner, fully managed HTTPS servers - Full control over almost every aspect of the system - HTTP->HTTPS redirects - Solves all 3 ACME challenges: HTTP, TLS-ALPN, and DNS - Most robust error handling of _any_ ACME client - Challenges are randomized to avoid accidental dependence - Challenges are rotated to overcome certain network blockages - Robust retries for up to 30 days - Exponential backoff with carefully-tuned intervals - Retries with optional test/staging CA endpoint instead of production, to avoid rate limits - Written in Go, a language with memory-safety guarantees - Powered by [ACMEz](https://github.com/mholt/acmez), _the_ premier ACME client library for Go - All [libdns](https://github.com/libdns) DNS providers work out-of-the-box - Pluggable storage implementations (default: file system) - Wildcard certificates - Automatic OCSP stapling ([done right](https://gist.github.com/sleevi/5efe9ef98961ecfb4da8#gistcomment-2336055)) [keeps your sites online!](https://twitter.com/caddyserver/status/1234874273724084226) - Will [automatically attempt](https://twitter.com/mholt6/status/1235577699541762048) to replace [revoked certificates](https://community.letsencrypt.org/t/2020-02-29-caa-rechecking-bug/114591/3?u=mholt)! - Staples stored to disk in case of responder outages - Distributed solving of all challenges (works behind load balancers) - Highly efficient, coordinated management in a fleet - Active locking - Smart queueing - Supports "on-demand" issuance of certificates (during TLS handshakes!) - Caddy / CertMagic pioneered this technology - Custom decision functions to regulate and throttle on-demand behavior - Optional event hooks for observation - Works with any certificate authority (CA) compliant with the ACME specification - Certificate revocation (please, only if private key is compromised) - Must-Staple (optional; not default) - Cross-platform support! Mac, Windows, Linux, BSD, Android... - Scales to hundreds of thousands of names/certificates per instance - Use in conjunction with your own certificates ## Requirements 0. ACME server (can be a publicly-trusted CA, or your own) 1. Public DNS name(s) you control 2. Server reachable from public Internet - Or use the DNS challenge to waive this requirement 3. Control over port 80 (HTTP) and/or 443 (HTTPS) - Or they can be forwarded to other ports you control - Or use the DNS challenge to waive this requirement - (This is a requirement of the ACME protocol, not a library limitation) 4. Persistent storage - Typically the local file system (default) - Other integrations available/possible **_Before using this library, your domain names MUST be pointed (A/AAAA records) at your server (unless you use the DNS challenge)!_** ## Installation ```bash $ go get github.com/caddyserver/certmagic ``` ## Usage ### Package Overview #### Certificate authority This library uses Let's Encrypt by default, but you can use any certificate authority that conforms to the ACME specification. Known/common CAs are provided as consts in the package, for example `LetsEncryptStagingCA` and `LetsEncryptProductionCA`. #### The `Config` type The `certmagic.Config` struct is how you can wield the power of this fully armed and operational battle station. However, an empty/uninitialized `Config` is _not_ a valid one! In time, you will learn to use the force of `certmagic.NewDefault()` as I have. #### Defaults The default `Config` value is called `certmagic.Default`. Change its fields to suit your needs, then call `certmagic.NewDefault()` when you need a valid `Config` value. In other words, `certmagic.Default` is a template and is not valid for use directly. You can set the default values easily, for example: `certmagic.Default.Issuer = ...`. Similarly, to configure ACME-specific defaults, use `certmagic.DefaultACME`. The high-level functions in this package (`HTTPS()`, `Listen()`, `ManageSync()`, and `ManageAsync()`) use the default config exclusively. This is how most of you will interact with the package. This is suitable when all your certificates are managed the same way. However, if you need to manage certificates differently depending on their name, you will need to make your own cache and configs (keep reading). #### Providing an email address Although not strictly required, this is highly recommended best practice. It allows you to receive expiration emails if your certificates are expiring for some reason, and also allows the CA's engineers to potentially get in touch with you if something is wrong. I recommend setting `certmagic.DefaultACME.Email` or always setting the `Email` field of a new `Config` struct. #### Rate limiting To avoid firehosing the CA's servers, CertMagic has built-in rate limiting. Currently, its default limit is up to 10 transactions (obtain or renew) every 1 minute (sliding window). This can be changed by setting the `RateLimitEvents` and `RateLimitEventsWindow` variables, if desired. The CA may still enforce their own rate limits, and there's nothing (well, nothing ethical) CertMagic can do to bypass them for you. Additionally, CertMagic will retry failed validations with exponential backoff for up to 30 days, with a reasonable maximum interval between attempts (an "attempt" means trying each enabled challenge type once). ### Development and Testing Note that Let's Encrypt imposes [strict rate limits](https://letsencrypt.org/docs/rate-limits/) at its production endpoint, so using it while developing your application may lock you out for a few days if you aren't careful! While developing your application and testing it, use [their staging endpoint](https://letsencrypt.org/docs/staging-environment/) which has much higher rate limits. Even then, don't hammer it: but it's much safer for when you're testing. When deploying, though, use their production CA because their staging CA doesn't issue trusted certificates. To use staging, set `certmagic.DefaultACME.CA = certmagic.LetsEncryptStagingCA` or set `CA` of every `ACMEManager` struct. ### Examples There are many ways to use this library. We'll start with the highest-level (simplest) and work down (more control). All these high-level examples use `certmagic.Default` and `certmagic.DefaultACME` for the config and the default cache and storage for serving up certificates. First, we'll follow best practices and do the following: ```go // read and agree to your CA's legal documents certmagic.DefaultACME.Agreed = true // provide an email address certmagic.DefaultACME.Email = "you@yours.com" // use the staging endpoint while we're developing certmagic.DefaultACME.CA = certmagic.LetsEncryptStagingCA ``` For fully-functional program examples, check out [this Twitter thread](https://twitter.com/mholt6/status/1073103805112147968) (or read it [unrolled into a single post](https://threadreaderapp.com/thread/1073103805112147968.html)). (Note that the package API has changed slightly since these posts.) #### Serving HTTP handlers with HTTPS ```go err := certmagic.HTTPS([]string{"example.com", "www.example.com"}, mux) if err != nil { return err } ``` This starts HTTP and HTTPS listeners and redirects HTTP to HTTPS! #### Starting a TLS listener ```go ln, err := certmagic.Listen([]string{"example.com"}) if err != nil { return err } ``` #### Getting a tls.Config ```go tlsConfig, err := certmagic.TLS([]string{"example.com"}) if err != nil { return err } ``` #### Advanced use For more control (particularly, if you need a different way of managing each certificate), you'll make and use a `Cache` and a `Config` like so: ```go cache := certmagic.NewCache(certmagic.CacheOptions{ GetConfigForCert: func(cert certmagic.Certificate) (*certmagic.Config, error) { // do whatever you need to do to get the right // configuration for this certificate; keep in // mind that this config value is used as a // template, and will be completed with any // defaults that are set in the Default config return &certmagic.Config{ // ... }, nil }, ... }) magic := certmagic.New(cache, certmagic.Config{ // any customizations you need go here }) myACME := certmagic.NewACMEManager(magic, certmagic.ACMEManager{ CA: certmagic.LetsEncryptStagingCA, Email: "you@yours.com", Agreed: true, // plus any other customizations you need }) magic.Issuer = myACME // this obtains certificates or renews them if necessary err := magic.ManageSync(context.TODO(), []string{"example.com", "sub.example.com"}) if err != nil { return err } // to use its certificates and solve the TLS-ALPN challenge, // you can get a TLS config to use in a TLS listener! tlsConfig := magic.TLSConfig() // be sure to customize NextProtos if serving a specific // application protocol after the TLS handshake, for example: tlsConfig.NextProtos = append([]string{"h2", "http/1.1"}, tlsConfig.NextProtos...) //// OR //// // if you already have a TLS config you don't want to replace, // we can simply set its GetCertificate field and append the // TLS-ALPN challenge protocol to the NextProtos myTLSConfig.GetCertificate = magic.GetCertificate myTLSConfig.NextProtos = append(myTLSConfig.NextProtos, tlsalpn01.ACMETLS1Protocol) // the HTTP challenge has to be handled by your HTTP server; // if you don't have one, you should have disabled it earlier // when you made the certmagic.Config httpMux = myACME.HTTPChallengeHandler(httpMux) ``` Great! This example grants you much more flexibility for advanced programs. However, _the vast majority of you will only use the high-level functions described earlier_, especially since you can still customize them by setting the package-level `Default` config. ### Wildcard certificates At time of writing (December 2018), Let's Encrypt only issues wildcard certificates with the DNS challenge. You can easily enable the DNS challenge with CertMagic for numerous providers (see the relevant section in the docs). ### Behind a load balancer (or in a cluster) CertMagic runs effectively behind load balancers and/or in cluster/fleet environments. In other words, you can have 10 or 1,000 servers all serving the same domain names, all sharing certificates and OCSP staples. To do so, simply ensure that each instance is using the same Storage. That is the sole criteria for determining whether an instance is part of a cluster. The default Storage is implemented using the file system, so mounting the same shared folder is sufficient (see [Storage](#storage) for more on that)! If you need an alternate Storage implementation, feel free to use one, provided that all the instances use the _same_ one. :) See [Storage](#storage) and the associated [pkg.go.dev](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Storage) for more information! ## The ACME Challenges This section describes how to solve the ACME challenges. Challenges are how you demonstrate to the certificate authority some control over your domain name, thus authorizing them to grant you a certificate for that name. [The great innovation of ACME](https://www.dotconferences.com/2016/10/matthew-holt-go-with-acme) is that verification by CAs can now be automated, rather than having to click links in emails (who ever thought that was a good idea??). If you're using the high-level convenience functions like `HTTPS()`, `Listen()`, or `TLS()`, the HTTP and/or TLS-ALPN challenges are solved for you because they also start listeners. However, if you're making a `Config` and you start your own server manually, you'll need to be sure the ACME challenges can be solved so certificates can be renewed. The HTTP and TLS-ALPN challenges are the defaults because they don't require configuration from you, but they require that your server is accessible from external IPs on low ports. If that is not possible in your situation, you can enable the DNS challenge, which will disable the HTTP and TLS-ALPN challenges and use the DNS challenge exclusively. Technically, only one challenge needs to be enabled for things to work, but using multiple is good for reliability in case a challenge is discontinued by the CA. This happened to the TLS-SNI challenge in early 2018—many popular ACME clients such as Traefik and Autocert broke, resulting in downtime for some sites, until new releases were made and patches deployed, because they used only one challenge; Caddy, however—this library's forerunner—was unaffected because it also used the HTTP challenge. If multiple challenges are enabled, they are chosen randomly to help prevent false reliance on a single challenge type. And if one fails, any remaining enabled challenges are tried before giving up. ### HTTP Challenge Per the ACME spec, the HTTP challenge requires port 80, or at least packet forwarding from port 80. It works by serving a specific HTTP response that only the genuine server would have to a normal HTTP request at a special endpoint. If you are running an HTTP server, solving this challenge is very easy: just wrap your handler in `HTTPChallengeHandler` _or_ call `SolveHTTPChallenge()` inside your own `ServeHTTP()` method. For example, if you're using the standard library: ```go mux := http.NewServeMux() mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { fmt.Fprintf(w, "Lookit my cool website over HTTPS!") }) http.ListenAndServe(":80", myACME.HTTPChallengeHandler(mux)) ``` If wrapping your handler is not a good solution, try this inside your `ServeHTTP()` instead: ```go magic := certmagic.NewDefault() myACME := certmagic.NewACMEManager(magic, certmagic.DefaultACME) func ServeHTTP(w http.ResponseWriter, req *http.Request) { if myACME.HandleHTTPChallenge(w, r) { return // challenge handled; nothing else to do } ... } ``` If you are not running an HTTP server, you should disable the HTTP challenge _or_ run an HTTP server whose sole job it is to solve the HTTP challenge. ### TLS-ALPN Challenge Per the ACME spec, the TLS-ALPN challenge requires port 443, or at least packet forwarding from port 443. It works by providing a special certificate using a standard TLS extension, Application Layer Protocol Negotiation (ALPN), having a special value. This is the most convenient challenge type because it usually requires no extra configuration and uses the standard TLS port which is where the certificates are used, also. This challenge is easy to solve: just use the provided `tls.Config` when you make your TLS listener: ```go // use this to configure a TLS listener tlsConfig := magic.TLSConfig() ``` Or make two simple changes to an existing `tls.Config`: ```go myTLSConfig.GetCertificate = magic.GetCertificate myTLSConfig.NextProtos = append(myTLSConfig.NextProtos, tlsalpn01.ACMETLS1Protocol} ``` Then just make sure your TLS listener is listening on port 443: ```go ln, err := tls.Listen("tcp", ":443", myTLSConfig) ``` ### DNS Challenge The DNS challenge is perhaps the most useful challenge because it allows you to obtain certificates without your server needing to be publicly accessible on the Internet, and it's the only challenge by which Let's Encrypt will issue wildcard certificates. This challenge works by setting a special record in the domain's zone. To do this automatically, your DNS provider needs to offer an API by which changes can be made to domain names, and the changes need to take effect immediately for best results. CertMagic supports [all DNS providers with `libdns` implementations](https://github.com/libdns)! It always cleans up the temporary record after the challenge completes. To enable it, just set the `DNS01Solver` field on a `certmagic.ACMEManager` struct, or set the default `certmagic.ACMEManager.DNS01Solver` variable. For example, if my domains' DNS was served by Cloudflare: ```go import "github.com/libdns/cloudflare" certmagic.DefaultACME.DNS01Solver = &certmagic.DNS01Solver{ DNSProvider: &cloudflare.Provider{ APIToken: "topsecret", }, } ``` Now the DNS challenge will be used by default, and I can obtain certificates for wildcard domains, too. Enabling the DNS challenge disables the other challenges for that `certmagic.ACMEManager` instance. ## On-Demand TLS Normally, certificates are obtained and renewed before a listener starts serving, and then those certificates are maintained throughout the lifetime of the program. In other words, the certificate names are static. But sometimes you don't know all the names ahead of time, or you don't want to manage all the certificates up front. This is where On-Demand TLS shines. Originally invented for use in Caddy (which was the first program to use such technology), On-Demand TLS makes it possible and easy to serve certificates for arbitrary or specific names during the lifetime of the server. When a TLS handshake is received, CertMagic will read the Server Name Indication (SNI) value and either load and present that certificate in the ServerHello, or if one does not exist, it will obtain it from a CA right then-and-there. Of course, this has some obvious security implications. You don't want to DoS a CA or allow arbitrary clients to fill your storage with spammy TLS handshakes. That's why, when you enable On-Demand issuance, you should set limits or policy to allow getting certificates. CertMagic has an implicit whitelist built-in which is sufficient for nearly everyone, but also has a more advanced way to control on-demand issuance. The simplest way to enable on-demand issuance is to set the OnDemand field of a Config (or the default package-level value): ```go certmagic.Default.OnDemand = new(certmagic.OnDemandConfig) ``` By setting this to a non-nil value, on-demand TLS is enabled for that config. For convenient security, CertMagic's high-level abstraction functions such as `HTTPS()`, `TLS()`, `ManageSync()`, `ManageAsync()`, and `Listen()` (which all accept a list of domain names) will whitelist those names automatically so only certificates for those names can be obtained when using the Default config. Usually this is sufficient for most users. However, if you require advanced control over which domains can be issued certificates on-demand (for example, if you do not know which domain names you are managing, or just need to defer their operations until later), you should implement your own DecisionFunc: ```go // if the decision function returns an error, a certificate // may not be obtained for that name at that time certmagic.Default.OnDemand = &certmagic.OnDemandConfig{ DecisionFunc: func(name string) error { if name != "example.com" { return fmt.Errorf("not allowed") } return nil }, } ``` The [pkg.go.dev](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#OnDemandConfig) describes how to use this in full detail, so please check it out! ## Storage CertMagic relies on storage to store certificates and other TLS assets (OCSP staple cache, coordinating locks, etc). Persistent storage is a requirement when using CertMagic: ephemeral storage will likely lead to rate limiting on the CA-side as CertMagic will always have to get new certificates. By default, CertMagic stores assets on the local file system in `$HOME/.local/share/certmagic` (and honors `$XDG_DATA_HOME` if set). CertMagic will create the directory if it does not exist. If writes are denied, things will not be happy, so make sure CertMagic can write to it! The notion of a "cluster" or "fleet" of instances that may be serving the same site and sharing certificates, etc, is tied to storage. Simply, any instances that use the same storage facilities are considered part of the cluster. So if you deploy 100 instances of CertMagic behind a load balancer, they are all part of the same cluster if they share the same storage configuration. Sharing storage could be mounting a shared folder, or implementing some other distributed storage system such as a database server or KV store. The easiest way to change the storage being used is to set `certmagic.DefaultStorage` to a value that satisfies the [Storage interface](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Storage). Keep in mind that a valid `Storage` must be able to implement some operations atomically in order to provide locking and synchronization. If you write a Storage implementation, please add it to the [project wiki](https://github.com/caddyserver/certmagic/wiki/Storage-Implementations) so people can find it! ## Cache All of the certificates in use are de-duplicated and cached in memory for optimal performance at handshake-time. This cache must be backed by persistent storage as described above. Most applications will not need to interact with certificate caches directly. Usually, the closest you will come is to set the package-wide `certmagic.DefaultStorage` variable (before attempting to create any Configs). However, if your use case requires using different storage facilities for different Configs (that's highly unlikely and NOT recommended! Even Caddy doesn't get that crazy), you will need to call `certmagic.NewCache()` and pass in the storage you want to use, then get new `Config` structs with `certmagic.NewWithCache()` and pass in the cache. Again, if you're needing to do this, you've probably over-complicated your application design. ## FAQ ### Can I use some of my own certificates while using CertMagic? Yes, just call the relevant method on the `Config` to add your own certificate to the cache: - [`CacheUnmanagedCertificatePEMBytes()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedCertificatePEMBytes) - [`CacheUnmanagedCertificatePEMFile()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedCertificatePEMFile) - [`CacheUnmanagedTLSCertificate()`](https://pkg.go.dev/github.com/caddyserver/certmagic?tab=doc#Config.CacheUnmanagedTLSCertificate) Keep in mind that unmanaged certificates are (obviously) not renewed for you, so you'll have to replace them when you do. However, OCSP stapling is performed even for unmanaged certificates that qualify. ### Does CertMagic obtain SAN certificates? Technically all certificates these days are SAN certificates because CommonName is deprecated. But if you're asking whether CertMagic issues and manages certificates with multiple SANs, the answer is no. But it does support serving them, if you provide your own. ### How can I listen on ports 80 and 443? Do I have to run as root? On Linux, you can use `setcap` to grant your binary the permission to bind low ports: ```bash $ sudo setcap cap_net_bind_service=+ep /path/to/your/binary ``` and then you will not need to run with root privileges. ## Contributing We welcome your contributions! Please see our **[contributing guidelines](https://github.com/caddyserver/certmagic/blob/master/.github/CONTRIBUTING.md)** for instructions. ## Project History CertMagic is the core of Caddy's advanced TLS automation code, extracted into a library. The underlying ACME client implementation is [ACMEz](https://github.com/mholt/acmez). CertMagic's code was originally a central part of Caddy even before Let's Encrypt entered public beta in 2015. In the years since then, Caddy's TLS automation techniques have been widely adopted, tried and tested in production, and served millions of sites and secured trillions of connections. Now, CertMagic is _the actual library used by Caddy_. It's incredibly powerful and feature-rich, but also easy to use for simple Go programs: one line of code can enable fully-automated HTTPS applications with HTTP->HTTPS redirects. Caddy is known for its robust HTTPS+ACME features. When ACME certificate authorities have had outages, in some cases Caddy was the only major client that didn't experience any downtime. Caddy can weather OCSP outages lasting days, or CA outages lasting weeks, without taking your sites offline. Caddy was also the first to sport "on-demand" issuance technology, which obtains certificates during the first TLS handshake for an allowed SNI name. Consequently, CertMagic brings all these (and more) features and capabilities right into your own Go programs. You can [watch a 2016 dotGo talk](https://www.dotconferences.com/2016/10/matthew-holt-go-with-acme) by the author of this library about using ACME to automate certificate management in Go programs: [![Matthew Holt speaking at dotGo 2016 about ACME in Go](https://user-images.githubusercontent.com/1128849/49921557-2d506780-fe6b-11e8-97bf-6053b6b4eb48.png)](https://www.dotconferences.com/2016/10/matthew-holt-go-with-acme) ## Credits and License CertMagic is a project by [Matthew Holt](https://twitter.com/mholt6), who is the author; and various contributors, who are credited in the commit history of either CertMagic or Caddy. CertMagic is licensed under Apache 2.0, an open source license. For convenience, its main points are summarized as follows (but this is no replacement for the actual license text): - The author owns the copyright to this code - Use, distribute, and modify the software freely - Private and internal use is allowed - License text and copyright notices must stay intact and be included with distributions - Any and all changes to the code must be documented golang-github-caddyserver-certmagic-0.15.2/account.go000066400000000000000000000321661414454377300226020ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bufio" "bytes" "context" "crypto/ecdsa" "crypto/elliptic" "crypto/rand" "encoding/json" "fmt" "io" "os" "path" "sort" "strings" "sync" "github.com/mholt/acmez/acme" ) // getAccount either loads or creates a new account, depending on if // an account can be found in storage for the given CA + email combo. func (am *ACMEManager) getAccount(ca, email string) (acme.Account, error) { acct, err := am.loadAccount(ca, email) if err != nil { if _, ok := err.(ErrNotExist); ok { return am.newAccount(email) } return acct, err } return acct, err } // loadAccount loads an account from storage, but does not create a new one. func (am *ACMEManager) loadAccount(ca, email string) (acme.Account, error) { regBytes, err := am.config.Storage.Load(am.storageKeyUserReg(ca, email)) if err != nil { return acme.Account{}, err } keyBytes, err := am.config.Storage.Load(am.storageKeyUserPrivateKey(ca, email)) if err != nil { return acme.Account{}, err } var acct acme.Account err = json.Unmarshal(regBytes, &acct) if err != nil { return acct, err } acct.PrivateKey, err = decodePrivateKey(keyBytes) if err != nil { return acct, fmt.Errorf("could not decode account's private key: %v", err) } return acct, nil } // newAccount generates a new private key for a new ACME account, but // it does not register or save the account. func (*ACMEManager) newAccount(email string) (acme.Account, error) { var acct acme.Account if email != "" { acct.Contact = []string{"mailto:" + email} // TODO: should we abstract the contact scheme? } privateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) if err != nil { return acct, fmt.Errorf("generating private key: %v", err) } acct.PrivateKey = privateKey return acct, nil } // GetAccount first tries loading the account with the associated private key from storage. // If it does not exist in storage, it will be retrieved from the ACME server and added to storage. // The account must already exist; it does not create a new account. func (am *ACMEManager) GetAccount(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { account, err := am.loadAccountByKey(ctx, privateKeyPEM) if err != nil { if _, ok := err.(ErrNotExist); ok { account, err = am.lookUpAccount(ctx, privateKeyPEM) } else { return account, err } } return account, err } // loadAccountByKey loads the account with the given private key from storage, if it exists. // If it does not exist, an error of type ErrNotExist is returned. This is not very efficient // for lots of accounts. func (am *ACMEManager) loadAccountByKey(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { accountList, err := am.config.Storage.List(am.storageKeyUsersPrefix(am.CA), false) if err != nil { return acme.Account{}, err } for _, accountFolderKey := range accountList { email := path.Base(accountFolderKey) keyBytes, err := am.config.Storage.Load(am.storageKeyUserPrivateKey(am.CA, email)) if err != nil { return acme.Account{}, err } if bytes.Equal(bytes.TrimSpace(keyBytes), bytes.TrimSpace(privateKeyPEM)) { return am.loadAccount(am.CA, email) } } return acme.Account{}, ErrNotExist(fmt.Errorf("no account found with that key")) } // lookUpAccount looks up the account associated with privateKeyPEM from the ACME server. // If the account is found by the server, it will be saved to storage and returned. func (am *ACMEManager) lookUpAccount(ctx context.Context, privateKeyPEM []byte) (acme.Account, error) { client, err := am.newACMEClient(false) if err != nil { return acme.Account{}, fmt.Errorf("creating ACME client: %v", err) } privateKey, err := decodePrivateKey([]byte(privateKeyPEM)) if err != nil { return acme.Account{}, fmt.Errorf("decoding private key: %v", err) } // look up the account account := acme.Account{PrivateKey: privateKey} account, err = client.GetAccount(ctx, account) if err != nil { return acme.Account{}, fmt.Errorf("looking up account with server: %v", err) } // save the account details to storage err = am.saveAccount(client.Directory, account) if err != nil { return account, fmt.Errorf("could not save account to storage: %v", err) } return account, nil } // saveAccount persists an ACME account's info and private key to storage. // It does NOT register the account via ACME or prompt the user. func (am *ACMEManager) saveAccount(ca string, account acme.Account) error { regBytes, err := json.MarshalIndent(account, "", "\t") if err != nil { return err } keyBytes, err := encodePrivateKey(account.PrivateKey) if err != nil { return err } // extract primary contact (email), without scheme (e.g. "mailto:") primaryContact := getPrimaryContact(account) all := []keyValue{ { key: am.storageKeyUserReg(ca, primaryContact), value: regBytes, }, { key: am.storageKeyUserPrivateKey(ca, primaryContact), value: keyBytes, }, } return storeTx(am.config.Storage, all) } // getEmail does everything it can to obtain an email address // from the user within the scope of memory and storage to use // for ACME TLS. If it cannot get an email address, it does nothing // (If user is prompted, it will warn the user of // the consequences of an empty email.) This function MAY prompt // the user for input. If allowPrompts is false, the user // will NOT be prompted and an empty email may be returned. func (am *ACMEManager) getEmail(allowPrompts bool) error { leEmail := am.Email // First try package default email, or a discovered email address if leEmail == "" { leEmail = DefaultACME.Email } if leEmail == "" { discoveredEmailMu.Lock() leEmail = discoveredEmail discoveredEmailMu.Unlock() } // Then try to get most recent user email from storage var gotRecentEmail bool if leEmail == "" { leEmail, gotRecentEmail = am.mostRecentAccountEmail(am.CA) } if !gotRecentEmail && leEmail == "" && allowPrompts { // Looks like there is no email address readily available, // so we will have to ask the user if we can. var err error leEmail, err = am.promptUserForEmail() if err != nil { return err } // User might have just signified their agreement am.Agreed = DefaultACME.Agreed } // save the email for later and ensure it is consistent // for repeated use; then update cfg with the email leEmail = strings.TrimSpace(strings.ToLower(leEmail)) discoveredEmailMu.Lock() if discoveredEmail == "" { discoveredEmail = leEmail } discoveredEmailMu.Unlock() am.Email = leEmail return nil } // promptUserForEmail prompts the user for an email address // and returns the email address they entered (which could // be the empty string). If no error is returned, then Agreed // will also be set to true, since continuing through the // prompt signifies agreement. func (am *ACMEManager) promptUserForEmail() (string, error) { // prompt the user for an email address and terms agreement reader := bufio.NewReader(stdin) am.promptUserAgreement("") fmt.Println("Please enter your email address to signify agreement and to be notified") fmt.Println("in case of issues. You can leave it blank, but we don't recommend it.") fmt.Print(" Email address: ") leEmail, err := reader.ReadString('\n') if err != nil && err != io.EOF { return "", fmt.Errorf("reading email address: %v", err) } leEmail = strings.TrimSpace(leEmail) DefaultACME.Agreed = true return leEmail, nil } // promptUserAgreement simply outputs the standard user // agreement prompt with the given agreement URL. // It outputs a newline after the message. func (am *ACMEManager) promptUserAgreement(agreementURL string) { userAgreementPrompt := `Your sites will be served over HTTPS automatically using an automated CA. By continuing, you agree to the CA's terms of service` if agreementURL == "" { fmt.Printf("\n\n%s.\n", userAgreementPrompt) return } fmt.Printf("\n\n%s at:\n %s\n", userAgreementPrompt, agreementURL) } // askUserAgreement prompts the user to agree to the agreement // at the given agreement URL via stdin. It returns whether the // user agreed or not. func (am *ACMEManager) askUserAgreement(agreementURL string) bool { am.promptUserAgreement(agreementURL) fmt.Print("Do you agree to the terms? (y/n): ") reader := bufio.NewReader(stdin) answer, err := reader.ReadString('\n') if err != nil { return false } answer = strings.ToLower(strings.TrimSpace(answer)) return answer == "y" || answer == "yes" } func storageKeyACMECAPrefix(issuerKey string) string { return path.Join(prefixACME, StorageKeys.Safe(issuerKey)) } func (am *ACMEManager) storageKeyCAPrefix(caURL string) string { return storageKeyACMECAPrefix(am.issuerKey(caURL)) } func (am *ACMEManager) storageKeyUsersPrefix(caURL string) string { return path.Join(am.storageKeyCAPrefix(caURL), "users") } func (am *ACMEManager) storageKeyUserPrefix(caURL, email string) string { if email == "" { email = emptyEmail } return path.Join(am.storageKeyUsersPrefix(caURL), StorageKeys.Safe(email)) } func (am *ACMEManager) storageKeyUserReg(caURL, email string) string { return am.storageSafeUserKey(caURL, email, "registration", ".json") } func (am *ACMEManager) storageKeyUserPrivateKey(caURL, email string) string { return am.storageSafeUserKey(caURL, email, "private", ".key") } // storageSafeUserKey returns a key for the given email, with the default // filename, and the filename ending in the given extension. func (am *ACMEManager) storageSafeUserKey(ca, email, defaultFilename, extension string) string { if email == "" { email = emptyEmail } email = strings.ToLower(email) filename := am.emailUsername(email) if filename == "" { filename = defaultFilename } filename = StorageKeys.Safe(filename) return path.Join(am.storageKeyUserPrefix(ca, email), filename+extension) } // emailUsername returns the username portion of an email address (part before // '@') or the original input if it can't find the "@" symbol. func (*ACMEManager) emailUsername(email string) string { at := strings.Index(email, "@") if at == -1 { return email } else if at == 0 { return email[1:] } return email[:at] } // mostRecentAccountEmail finds the most recently-written account file // in storage. Since this is part of a complex sequence to get a user // account, errors here are discarded to simplify code flow in // the caller, and errors are not important here anyway. func (am *ACMEManager) mostRecentAccountEmail(caURL string) (string, bool) { accountList, err := am.config.Storage.List(am.storageKeyUsersPrefix(caURL), false) if err != nil || len(accountList) == 0 { return "", false } // get all the key infos ahead of sorting, because // we might filter some out stats := make(map[string]KeyInfo) for i := 0; i < len(accountList); i++ { u := accountList[i] keyInfo, err := am.config.Storage.Stat(u) if err != nil { continue } if keyInfo.IsTerminal { // I found a bug when macOS created a .DS_Store file in // the users folder, and CertMagic tried to use that as // the user email because it was newer than the other one // which existed... sure, this isn't a perfect fix but // frankly one's OS shouldn't mess with the data folder // in the first place. accountList = append(accountList[:i], accountList[i+1:]...) i-- continue } stats[u] = keyInfo } sort.Slice(accountList, func(i, j int) bool { iInfo := stats[accountList[i]] jInfo := stats[accountList[j]] return jInfo.Modified.Before(iInfo.Modified) }) if len(accountList) == 0 { return "", false } account, err := am.getAccount(caURL, path.Base(accountList[0])) if err != nil { return "", false } return getPrimaryContact(account), true } // getPrimaryContact returns the first contact on the account (if any) // without the scheme. (I guess we assume an email address.) func getPrimaryContact(account acme.Account) string { // TODO: should this be abstracted with some lower-level helper? var primaryContact string if len(account.Contact) > 0 { primaryContact = account.Contact[0] if idx := strings.Index(primaryContact, ":"); idx >= 0 { primaryContact = primaryContact[idx+1:] } } return primaryContact } // When an email address is not explicitly specified, we can remember // the last one we discovered to avoid having to ask again later. // (We used to store this in DefaultACME.Email but it was racey; see #127) var ( discoveredEmail string discoveredEmailMu sync.Mutex ) // stdin is used to read the user's input if prompted; // this is changed by tests during tests. var stdin = io.ReadWriter(os.Stdin) // The name of the folder for accounts where the email // address was not provided; default 'username' if you will, // but only for local/storage use, not with the CA. const emptyEmail = "default" golang-github-caddyserver-certmagic-0.15.2/account_test.go000066400000000000000000000157321414454377300236410ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "os" "path/filepath" "reflect" "strings" "testing" "time" ) func TestNewAccount(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, certCache: new(Cache), } am.config = testConfig email := "me@foobar.com" account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } if account.PrivateKey == nil { t.Error("Private key is nil") } if account.Contact[0] != "mailto:"+email { t.Errorf("Expected email to be %s, but was %s", email, account.Contact[0]) } if account.Status != "" { t.Error("New account already has a status; it shouldn't") } } func TestSaveAccount(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata1_tmp"}, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() email := "me@foobar.com" account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } _, err = am.getAccount(am.CA, email) if err != nil { t.Errorf("Cannot access account data, error: %v", err) } } func TestGetAccountDoesNotAlreadyExist(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, certCache: new(Cache), } am.config = testConfig account, err := am.getAccount(am.CA, "account_does_not_exist@foobar.com") if err != nil { t.Fatalf("Error getting account: %v", err) } if account.PrivateKey == nil { t.Error("Expected account to have a private key, but it was nil") } } func TestGetAccountAlreadyExists(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata2_tmp"}, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() email := "me@foobar.com" // Set up test account, err := am.newAccount(email) if err != nil { t.Fatalf("Error creating account: %v", err) } err = am.saveAccount(am.CA, account) if err != nil { t.Fatalf("Error saving account: %v", err) } // Expect to load account from disk loadedAccount, err := am.getAccount(am.CA, email) if err != nil { t.Fatalf("Error getting account: %v", err) } // Assert keys are the same if !privateKeysSame(account.PrivateKey, loadedAccount.PrivateKey) { t.Error("Expected private key to be the same after loading, but it wasn't") } // Assert emails are the same if !reflect.DeepEqual(account.Contact, loadedAccount.Contact) { t.Errorf("Expected contacts to be equal, but was '%s' before and '%s' after loading", account.Contact, loadedAccount.Contact) } } func TestGetEmailFromPackageDefault(t *testing.T) { DefaultACME.Email = "tEsT2@foo.com" defer func() { DefaultACME.Email = "" discoveredEmail = "" }() am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata2_tmp"}, certCache: new(Cache), } am.config = testConfig err := am.getEmail(true) if err != nil { t.Fatalf("getEmail error: %v", err) } lowerEmail := strings.ToLower(DefaultACME.Email) if am.Email != lowerEmail { t.Errorf("Did not get correct email from memory; expected '%s' but got '%s'", lowerEmail, am.Email) } } func TestGetEmailFromUserInput(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata3_tmp"}, certCache: new(Cache), } am.config = testConfig // let's not clutter up the output origStdout := os.Stdout os.Stdout = nil agreementTestURL = "(none - testing)" defer func() { os.Stdout = origStdout agreementTestURL = "" }() email := "test3@foo.com" stdin = bytes.NewBufferString(email + "\n") err := am.getEmail(true) if err != nil { t.Fatalf("getEmail error: %v", err) } if am.Email != email { t.Errorf("Did not get correct email from user input prompt; expected '%s' but got '%s'", email, am.Email) } if !am.Agreed { t.Error("Expect Config.Agreed to be true, but got false") } } func TestGetEmailFromRecent(t *testing.T) { am := &ACMEManager{CA: dummyCA} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata4_tmp"}, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() DefaultACME.Email = "" discoveredEmail = "" for i, eml := range []string{ "test4-1@foo.com", "test4-2@foo.com", "TEST4-3@foo.com", // test case insensitivity } { account, err := am.newAccount(eml) if err != nil { t.Fatalf("Error creating user %d: %v", i, err) } err = am.saveAccount(am.CA, account) if err != nil { t.Fatalf("Error saving user %d: %v", i, err) } // Change modified time so they're all different and the test becomes more deterministic fs := testConfig.Storage.(*FileStorage) userFolder := filepath.Join(fs.Path, am.storageKeyUserPrefix(am.CA, eml)) f, err := os.Stat(userFolder) if err != nil { t.Fatalf("Could not access user folder for '%s': %v", eml, err) } chTime := f.ModTime().Add(time.Duration(i) * time.Hour) // 1 second isn't always enough spacing! if err := os.Chtimes(userFolder, chTime, chTime); err != nil { t.Fatalf("Could not change user folder mod time for '%s': %v", eml, err) } } err := am.getEmail(true) if err != nil { t.Fatalf("getEmail error: %v", err) } if am.Email != "test4-3@foo.com" { t.Errorf("Did not get correct email from storage; expected '%s' but got '%s'", "test4-3@foo.com", am.Email) } } // agreementTestURL is set during tests to skip requiring // setting up an entire ACME CA endpoint. var agreementTestURL string golang-github-caddyserver-certmagic-0.15.2/acmeclient.go000066400000000000000000000274151414454377300232530ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "crypto/x509" "fmt" weakrand "math/rand" "net" "net/http" "net/url" "strconv" "strings" "sync" "time" "github.com/mholt/acmez" "github.com/mholt/acmez/acme" "go.uber.org/zap" ) func init() { weakrand.Seed(time.Now().UnixNano()) } // acmeClient holds state necessary to perform ACME operations // for certificate management with an ACME account. Call // ACMEManager.newACMEClientWithAccount() to get a valid one. type acmeClient struct { mgr *ACMEManager acmeClient *acmez.Client account acme.Account } // newACMEClientWithAccount creates an ACME client ready to use with an account, including // loading one from storage or registering a new account with the CA if necessary. If // useTestCA is true, am.TestCA will be used if set; otherwise, the primary CA will be used. func (am *ACMEManager) newACMEClientWithAccount(ctx context.Context, useTestCA, interactive bool) (*acmeClient, error) { // first, get underlying ACME client client, err := am.newACMEClient(useTestCA) if err != nil { return nil, err } // look up or create the ACME account var account acme.Account if am.AccountKeyPEM != "" { account, err = am.GetAccount(ctx, []byte(am.AccountKeyPEM)) } else { account, err = am.getAccount(client.Directory, am.Email) } if err != nil { return nil, fmt.Errorf("getting ACME account: %v", err) } // register account if it is new if account.Status == "" { if am.NewAccountFunc != nil { account, err = am.NewAccountFunc(ctx, am, account) if err != nil { return nil, fmt.Errorf("account pre-registration callback: %v", err) } } // agree to terms if interactive { if !am.Agreed { var termsURL string dir, err := client.GetDirectory(ctx) if err != nil { return nil, fmt.Errorf("getting directory: %w", err) } if dir.Meta != nil { termsURL = dir.Meta.TermsOfService } if termsURL != "" { am.Agreed = am.askUserAgreement(termsURL) if !am.Agreed { return nil, fmt.Errorf("user must agree to CA terms") } } } } else { // can't prompt a user who isn't there; they should // have reviewed the terms beforehand am.Agreed = true } account.TermsOfServiceAgreed = am.Agreed // associate account with external binding, if configured if am.ExternalAccount != nil { err := account.SetExternalAccountBinding(ctx, client.Client, *am.ExternalAccount) if err != nil { return nil, err } } // create account account, err = client.NewAccount(ctx, account) if err != nil { return nil, fmt.Errorf("registering account %v with server: %w", account.Contact, err) } // persist the account to storage err = am.saveAccount(client.Directory, account) if err != nil { return nil, fmt.Errorf("could not save account %v: %v", account.Contact, err) } } c := &acmeClient{ mgr: am, acmeClient: client, account: account, } return c, nil } // newACMEClient creates a new underlying ACME client using the settings in am, // independent of any particular ACME account. If useTestCA is true, am.TestCA // will be used if it is set; otherwise, the primary CA will be used. func (am *ACMEManager) newACMEClient(useTestCA bool) (*acmez.Client, error) { // ensure defaults are filled in var caURL string if useTestCA { caURL = am.TestCA } if caURL == "" { caURL = am.CA } if caURL == "" { caURL = DefaultACME.CA } certObtainTimeout := am.CertObtainTimeout if certObtainTimeout == 0 { certObtainTimeout = DefaultACME.CertObtainTimeout } // ensure endpoint is secure (assume HTTPS if scheme is missing) if !strings.Contains(caURL, "://") { caURL = "https://" + caURL } u, err := url.Parse(caURL) if err != nil { return nil, err } if u.Scheme != "https" && !isLoopback(u.Host) && !isInternal(u.Host) { return nil, fmt.Errorf("%s: insecure CA URL (HTTPS required)", caURL) } // set up the dialers and resolver for the ACME client's HTTP client dialer := &net.Dialer{ Timeout: 30 * time.Second, KeepAlive: 2 * time.Minute, } if am.Resolver != "" { dialer.Resolver = &net.Resolver{ PreferGo: true, Dial: func(ctx context.Context, network, _ string) (net.Conn, error) { return (&net.Dialer{ Timeout: 15 * time.Second, }).DialContext(ctx, network, am.Resolver) }, } } // TODO: we could potentially reuse the HTTP transport and client hc := am.httpClient // TODO: is this racey? if am.httpClient == nil { transport := &http.Transport{ Proxy: http.ProxyFromEnvironment, DialContext: dialer.DialContext, TLSHandshakeTimeout: 15 * time.Second, ResponseHeaderTimeout: 15 * time.Second, ExpectContinueTimeout: 2 * time.Second, ForceAttemptHTTP2: true, } if am.TrustedRoots != nil { transport.TLSClientConfig = &tls.Config{ RootCAs: am.TrustedRoots, } } hc = &http.Client{ Transport: transport, Timeout: HTTPTimeout, } am.httpClient = hc } client := &acmez.Client{ Client: &acme.Client{ Directory: caURL, PollTimeout: certObtainTimeout, UserAgent: buildUAString(), HTTPClient: hc, }, ChallengeSolvers: make(map[string]acmez.Solver), } if am.Logger != nil { l := am.Logger.Named("acme_client") client.Client.Logger, client.Logger = l, l } // configure challenges (most of the time, DNS challenge is // exclusive of other ones because it is usually only used // in situations where the default challenges would fail) if am.DNS01Solver == nil { // enable HTTP-01 challenge if !am.DisableHTTPChallenge { useHTTPPort := HTTPChallengePort if HTTPPort > 0 && HTTPPort != HTTPChallengePort { useHTTPPort = HTTPPort } if am.AltHTTPPort > 0 { useHTTPPort = am.AltHTTPPort } client.ChallengeSolvers[acme.ChallengeTypeHTTP01] = distributedSolver{ storage: am.config.Storage, storageKeyIssuerPrefix: am.storageKeyCAPrefix(client.Directory), solver: &httpSolver{ acmeManager: am, address: net.JoinHostPort(am.ListenHost, strconv.Itoa(useHTTPPort)), }, } } // enable TLS-ALPN-01 challenge if !am.DisableTLSALPNChallenge { useTLSALPNPort := TLSALPNChallengePort if HTTPSPort > 0 && HTTPSPort != TLSALPNChallengePort { useTLSALPNPort = HTTPSPort } if am.AltTLSALPNPort > 0 { useTLSALPNPort = am.AltTLSALPNPort } client.ChallengeSolvers[acme.ChallengeTypeTLSALPN01] = distributedSolver{ storage: am.config.Storage, storageKeyIssuerPrefix: am.storageKeyCAPrefix(client.Directory), solver: &tlsALPNSolver{ config: am.config, address: net.JoinHostPort(am.ListenHost, strconv.Itoa(useTLSALPNPort)), }, } } } else { // use DNS challenge exclusively client.ChallengeSolvers[acme.ChallengeTypeDNS01] = am.DNS01Solver } // wrap solvers in our wrapper so that we can keep track of challenge // info: this is useful for solving challenges globally as a process; // for example, usually there is only one process that can solve the // HTTP and TLS-ALPN challenges, and only one server in that process // that can bind the necessary port(s), so if a server listening on // a different port needed a certificate, it would have to know about // the other server listening on that port, and somehow convey its // challenge info or share its config, but this isn't always feasible; // what the wrapper does is it accesses a global challenge memory so // that unrelated servers in this process can all solve each others' // challenges without having to know about each other - Caddy's admin // endpoint uses this functionality since it and the HTTP/TLS modules // do not know about each other // (doing this here in a separate loop ensures that even if we expose // solver config to users later, we will even wrap their own solvers) for name, solver := range client.ChallengeSolvers { client.ChallengeSolvers[name] = solverWrapper{solver} } return client, nil } func (c *acmeClient) throttle(ctx context.Context, names []string) error { // throttling is scoped to CA + account email rateLimiterKey := c.acmeClient.Directory + "," + c.mgr.Email rateLimitersMu.Lock() rl, ok := rateLimiters[rateLimiterKey] if !ok { rl = NewRateLimiter(RateLimitEvents, RateLimitEventsWindow) rateLimiters[rateLimiterKey] = rl // TODO: stop rate limiter when it is garbage-collected... } rateLimitersMu.Unlock() if c.mgr.Logger != nil { c.mgr.Logger.Info("waiting on internal rate limiter", zap.Strings("identifiers", names), zap.String("ca", c.acmeClient.Directory), zap.String("account", c.mgr.Email), ) } err := rl.Wait(ctx) if err != nil { return err } if c.mgr.Logger != nil { c.mgr.Logger.Info("done waiting on internal rate limiter", zap.Strings("identifiers", names), zap.String("ca", c.acmeClient.Directory), zap.String("account", c.mgr.Email), ) } return nil } func (c *acmeClient) usingTestCA() bool { return c.mgr.TestCA != "" && c.acmeClient.Directory == c.mgr.TestCA } func (c *acmeClient) revoke(ctx context.Context, cert *x509.Certificate, reason int) error { return c.acmeClient.RevokeCertificate(ctx, c.account, cert, c.account.PrivateKey, reason) } func buildUAString() string { ua := "CertMagic" if UserAgent != "" { ua = UserAgent + " " + ua } return ua } // These internal rate limits are designed to prevent accidentally // firehosing a CA's ACME endpoints. They are not intended to // replace or replicate the CA's actual rate limits. // // Let's Encrypt's rate limits can be found here: // https://letsencrypt.org/docs/rate-limits/ // // Currently (as of December 2019), Let's Encrypt's most relevant // rate limit for large deployments is 300 new orders per account // per 3 hours (on average, or best case, that's about 1 every 36 // seconds, or 2 every 72 seconds, etc.); but it's not reasonable // to try to assume that our internal state is the same as the CA's // (due to process restarts, config changes, failed validations, // etc.) and ultimately, only the CA's actual rate limiter is the // authority. Thus, our own rate limiters do not attempt to enforce // external rate limits. Doing so causes problems when the domains // are not in our control (i.e. serving customer sites) and/or lots // of domains fail validation: they clog our internal rate limiter // and nearly starve out (or at least slow down) the other domains // that need certificates. Failed transactions are already retried // with exponential backoff, so adding in rate limiting can slow // things down even more. // // Instead, the point of our internal rate limiter is to avoid // hammering the CA's endpoint when there are thousands or even // millions of certificates under management. Our goal is to // allow small bursts in a relatively short timeframe so as to // not block any one domain for too long, without unleashing // thousands of requests to the CA at once. var ( rateLimiters = make(map[string]*RingBufferRateLimiter) rateLimitersMu sync.RWMutex // RateLimitEvents is how many new events can be allowed // in RateLimitEventsWindow. RateLimitEvents = 10 // RateLimitEventsWindow is the size of the sliding // window that throttles events. RateLimitEventsWindow = 10 * time.Second ) // Some default values passed down to the underlying ACME client. var ( UserAgent string HTTPTimeout = 30 * time.Second ) golang-github-caddyserver-certmagic-0.15.2/acmemanager.go000066400000000000000000000355431414454377300234100ustar00rootroot00000000000000package certmagic import ( "context" "crypto/x509" "errors" "fmt" "net/http" "net/url" "sort" "strings" "time" "github.com/mholt/acmez" "github.com/mholt/acmez/acme" "go.uber.org/zap" ) // ACMEManager gets certificates using ACME. It implements the PreChecker, // Issuer, and Revoker interfaces. // // It is NOT VALID to use an ACMEManager without calling NewACMEManager(). // It fills in any default values from DefaultACME as well as setting up // internal state that is necessary for valid use. Always call // NewACMEManager() to get a valid ACMEManager value. type ACMEManager struct { // The endpoint of the directory for the ACME // CA we are to use CA string // TestCA is the endpoint of the directory for // an ACME CA to use to test domain validation, // but any certs obtained from this CA are // discarded TestCA string // The email address to use when creating or // selecting an existing ACME server account Email string // The PEM-encoded private key of the ACME // account to use; only needed if the account // is already created on the server and // can be looked up with the ACME protocol AccountKeyPEM string // Set to true if agreed to the CA's // subscriber agreement Agreed bool // An optional external account to associate // with this ACME account ExternalAccount *acme.EAB // Disable all HTTP challenges DisableHTTPChallenge bool // Disable all TLS-ALPN challenges DisableTLSALPNChallenge bool // The host (ONLY the host, not port) to listen // on if necessary to start a listener to solve // an ACME challenge ListenHost string // The alternate port to use for the ACME HTTP // challenge; if non-empty, this port will be // used instead of HTTPChallengePort to spin up // a listener for the HTTP challenge AltHTTPPort int // The alternate port to use for the ACME // TLS-ALPN challenge; the system must forward // TLSALPNChallengePort to this port for // challenge to succeed AltTLSALPNPort int // The solver for the dns-01 challenge; // usually this is a DNS01Solver value // from this package DNS01Solver acmez.Solver // TrustedRoots specifies a pool of root CA // certificates to trust when communicating // over a network to a peer. TrustedRoots *x509.CertPool // The maximum amount of time to allow for // obtaining a certificate. If empty, the // default from the underlying ACME lib is // used. If set, it must not be too low so // as to cancel challenges too early. CertObtainTimeout time.Duration // Address of custom DNS resolver to be used // when communicating with ACME server Resolver string // Callback function that is called before a // new ACME account is registered with the CA; // it allows for last-second config changes // of the ACMEManager and the Account. // (TODO: this feature is still EXPERIMENTAL and subject to change) NewAccountFunc func(context.Context, *ACMEManager, acme.Account) (acme.Account, error) // Preferences for selecting alternate // certificate chains PreferredChains ChainPreference // Set a logger to enable logging Logger *zap.Logger config *Config httpClient *http.Client } // NewACMEManager constructs a valid ACMEManager based on a template // configuration; any empty values will be filled in by defaults in // DefaultACME, and if any required values are still empty, sensible // defaults will be used. // // Typically, you'll create the Config first with New() or NewDefault(), // then call NewACMEManager(), then assign the return value to the Issuers // field of the Config. func NewACMEManager(cfg *Config, template ACMEManager) *ACMEManager { if cfg == nil { panic("cannot make valid ACMEManager without an associated CertMagic config") } if template.CA == "" { template.CA = DefaultACME.CA } if template.TestCA == "" && template.CA == DefaultACME.CA { // only use the default test CA if the CA is also // the default CA; no point in testing against // Let's Encrypt's staging server if we are not // using their production server too template.TestCA = DefaultACME.TestCA } if template.Email == "" { template.Email = DefaultACME.Email } if template.AccountKeyPEM == "" { template.AccountKeyPEM = DefaultACME.AccountKeyPEM } if !template.Agreed { template.Agreed = DefaultACME.Agreed } if template.ExternalAccount == nil { template.ExternalAccount = DefaultACME.ExternalAccount } if !template.DisableHTTPChallenge { template.DisableHTTPChallenge = DefaultACME.DisableHTTPChallenge } if !template.DisableTLSALPNChallenge { template.DisableTLSALPNChallenge = DefaultACME.DisableTLSALPNChallenge } if template.ListenHost == "" { template.ListenHost = DefaultACME.ListenHost } if template.AltHTTPPort == 0 { template.AltHTTPPort = DefaultACME.AltHTTPPort } if template.AltTLSALPNPort == 0 { template.AltTLSALPNPort = DefaultACME.AltTLSALPNPort } if template.DNS01Solver == nil { template.DNS01Solver = DefaultACME.DNS01Solver } if template.TrustedRoots == nil { template.TrustedRoots = DefaultACME.TrustedRoots } if template.CertObtainTimeout == 0 { template.CertObtainTimeout = DefaultACME.CertObtainTimeout } if template.Resolver == "" { template.Resolver = DefaultACME.Resolver } if template.NewAccountFunc == nil { template.NewAccountFunc = DefaultACME.NewAccountFunc } if template.Logger == nil { template.Logger = DefaultACME.Logger } template.config = cfg return &template } // IssuerKey returns the unique issuer key for the // confgured CA endpoint. func (am *ACMEManager) IssuerKey() string { return am.issuerKey(am.CA) } func (*ACMEManager) issuerKey(ca string) string { key := ca if caURL, err := url.Parse(key); err == nil { key = caURL.Host if caURL.Path != "" { // keep the path, but make sure it's a single // component (i.e. no forward slashes, and for // good measure, no backward slashes either) const hyphen = "-" repl := strings.NewReplacer( "/", hyphen, "\\", hyphen, ) path := strings.Trim(repl.Replace(caURL.Path), hyphen) if path != "" { key += hyphen + path } } } return key } // PreCheck performs a few simple checks before obtaining or // renewing a certificate with ACME, and returns whether this // batch is eligible for certificates if using Let's Encrypt. // It also ensures that an email address is available. func (am *ACMEManager) PreCheck(_ context.Context, names []string, interactive bool) error { publicCA := strings.Contains(am.CA, "api.letsencrypt.org") || strings.Contains(am.CA, "acme.zerossl.com") if publicCA { for _, name := range names { if !SubjectQualifiesForPublicCert(name) { return fmt.Errorf("subject does not qualify for a public certificate: %s", name) } } } return am.getEmail(interactive) } // Issue implements the Issuer interface. It obtains a certificate for the given csr using // the ACME configuration am. func (am *ACMEManager) Issue(ctx context.Context, csr *x509.CertificateRequest) (*IssuedCertificate, error) { if am.config == nil { panic("missing config pointer (must use NewACMEManager)") } var isRetry bool if attempts, ok := ctx.Value(AttemptsCtxKey).(*int); ok { isRetry = *attempts > 0 } cert, usedTestCA, err := am.doIssue(ctx, csr, isRetry) if err != nil { return nil, err } // important to note that usedTestCA is not necessarily the same as isRetry // (usedTestCA can be true if the main CA and the test CA happen to be the same) if isRetry && usedTestCA && am.CA != am.TestCA { // succeeded with testing endpoint, so try again with production endpoint // (only if the production endpoint is different from the testing endpoint) // TODO: This logic is imperfect and could benefit from some refinement. // The two CA endpoints likely have different states, which could cause one // to succeed and the other to fail, even if it's not a validation error. // Two common cases would be: // 1) Rate limiter state. This is more likely to cause prod to fail while // staging succeeds, since prod usually has tighter rate limits. Thus, if // initial attempt failed in prod due to rate limit, first retry (on staging) // might succeed, and then trying prod again right way would probably still // fail; normally this would terminate retries but the right thing to do in // this case is to back off and retry again later. We could refine this logic // to stick with the production endpoint on retries unless the error changes. // 2) Cached authorizations state. If a domain validates successfully with // one endpoint, but then the other endpoint is used, it might fail, e.g. if // DNS was just changed or is still propagating. In this case, the second CA // should continue to be retried with backoff, without switching back to the // other endpoint. This is more likely to happen if a user is testing with // the staging CA as the main CA, then changes their configuration once they // think they are ready for the production endpoint. cert, _, err = am.doIssue(ctx, csr, false) if err != nil { // succeeded with test CA but failed just now with the production CA; // either we are observing differing internal states of each CA that will // work out with time, or there is a bug/misconfiguration somewhere // externally; it is hard to tell which! one easy cue is whether the // error is specifically a 429 (Too Many Requests); if so, we should // probably keep retrying var problem acme.Problem if errors.As(err, &problem) { if problem.Status == http.StatusTooManyRequests { // DON'T abort retries; the test CA succeeded (even // if it's cached, it recently succeeded!) so we just // need to keep trying (with backoff) until this CA's // rate limits expire... // TODO: as mentioned in comment above, we would benefit // by pinning the main CA at this point instead of // needlessly retrying with the test CA first each time return nil, err } } return nil, ErrNoRetry{err} } } return cert, err } func (am *ACMEManager) doIssue(ctx context.Context, csr *x509.CertificateRequest, useTestCA bool) (*IssuedCertificate, bool, error) { client, err := am.newACMEClientWithAccount(ctx, useTestCA, false) if err != nil { return nil, false, err } usingTestCA := client.usingTestCA() nameSet := namesFromCSR(csr) if !useTestCA { if err := client.throttle(ctx, nameSet); err != nil { return nil, usingTestCA, err } } certChains, err := client.acmeClient.ObtainCertificateUsingCSR(ctx, client.account, csr) if err != nil { return nil, usingTestCA, fmt.Errorf("%v %w (ca=%s)", nameSet, err, client.acmeClient.Directory) } if len(certChains) == 0 { return nil, usingTestCA, fmt.Errorf("no certificate chains") } preferredChain := am.selectPreferredChain(certChains) ic := &IssuedCertificate{ Certificate: preferredChain.ChainPEM, Metadata: preferredChain, } return ic, usingTestCA, nil } // selectPreferredChain sorts and then filters the certificate chains to find the optimal // chain preferred by the client. If there's only one chain, that is returned without any // processing. If there are no matches, the first chain is returned. func (am *ACMEManager) selectPreferredChain(certChains []acme.Certificate) acme.Certificate { if len(certChains) == 1 { if am.Logger != nil && (len(am.PreferredChains.AnyCommonName) > 0 || len(am.PreferredChains.RootCommonName) > 0) { am.Logger.Debug("there is only one chain offered; selecting it regardless of preferences", zap.String("chain_url", certChains[0].URL)) } return certChains[0] } if am.PreferredChains.Smallest != nil { if *am.PreferredChains.Smallest { sort.Slice(certChains, func(i, j int) bool { return len(certChains[i].ChainPEM) < len(certChains[j].ChainPEM) }) } else { sort.Slice(certChains, func(i, j int) bool { return len(certChains[i].ChainPEM) > len(certChains[j].ChainPEM) }) } } if len(am.PreferredChains.AnyCommonName) > 0 || len(am.PreferredChains.RootCommonName) > 0 { // in order to inspect, we need to decode their PEM contents decodedChains := make([][]*x509.Certificate, len(certChains)) for i, chain := range certChains { certs, err := parseCertsFromPEMBundle(chain.ChainPEM) if err != nil { if am.Logger != nil { am.Logger.Error("unable to parse PEM certificate chain", zap.Int("chain", i), zap.Error(err)) } continue } decodedChains[i] = certs } if len(am.PreferredChains.AnyCommonName) > 0 { for _, prefAnyCN := range am.PreferredChains.AnyCommonName { for i, chain := range decodedChains { for _, cert := range chain { if cert.Issuer.CommonName == prefAnyCN { if am.Logger != nil { am.Logger.Debug("found preferred certificate chain by issuer common name", zap.String("preference", prefAnyCN), zap.Int("chain", i)) } return certChains[i] } } } } } if len(am.PreferredChains.RootCommonName) > 0 { for _, prefRootCN := range am.PreferredChains.RootCommonName { for i, chain := range decodedChains { if chain[len(chain)-1].Issuer.CommonName == prefRootCN { if am.Logger != nil { am.Logger.Debug("found preferred certificate chain by root common name", zap.String("preference", prefRootCN), zap.Int("chain", i)) } return certChains[i] } } } } if am.Logger != nil { am.Logger.Warn("did not find chain matching preferences; using first") } } return certChains[0] } // Revoke implements the Revoker interface. It revokes the given certificate. func (am *ACMEManager) Revoke(ctx context.Context, cert CertificateResource, reason int) error { client, err := am.newACMEClientWithAccount(ctx, false, false) if err != nil { return err } certs, err := parseCertsFromPEMBundle(cert.CertificatePEM) if err != nil { return err } return client.revoke(ctx, certs[0], reason) } // ChainPreference describes the client's preferred certificate chain, // useful if the CA offers alternate chains. The first matching chain // will be selected. type ChainPreference struct { // Prefer chains with the fewest number of bytes. Smallest *bool // Select first chain having a root with one of // these common names. RootCommonName []string // Select first chain that has any issuer with one // of these common names. AnyCommonName []string } // DefaultACME specifies default settings to use for ACMEManagers. // Using this value is optional but can be convenient. var DefaultACME = ACMEManager{ CA: LetsEncryptProductionCA, TestCA: LetsEncryptStagingCA, } // Some well-known CA endpoints available to use. const ( LetsEncryptStagingCA = "https://acme-staging-v02.api.letsencrypt.org/directory" LetsEncryptProductionCA = "https://acme-v02.api.letsencrypt.org/directory" ZeroSSLProductionCA = "https://acme.zerossl.com/v2/DV90" ) // prefixACME is the storage key prefix used for ACME-specific assets. const prefixACME = "acme" // Interface guards var ( _ PreChecker = (*ACMEManager)(nil) _ Issuer = (*ACMEManager)(nil) _ Revoker = (*ACMEManager)(nil) ) golang-github-caddyserver-certmagic-0.15.2/acmemanager_test.go000066400000000000000000000001101414454377300244250ustar00rootroot00000000000000package certmagic const dummyCA = "https://example.com/acme/directory" golang-github-caddyserver-certmagic-0.15.2/async.go000066400000000000000000000114231414454377300222540ustar00rootroot00000000000000package certmagic import ( "context" "errors" "log" "runtime" "sync" "time" "go.uber.org/zap" ) var jm = &jobManager{maxConcurrentJobs: 1000} type jobManager struct { mu sync.Mutex maxConcurrentJobs int activeWorkers int queue []namedJob names map[string]struct{} } type namedJob struct { name string job func() error logger *zap.Logger } // Submit enqueues the given job with the given name. If name is non-empty // and a job with the same name is already enqueued or running, this is a // no-op. If name is empty, no duplicate prevention will occur. The job // manager will then run this job as soon as it is able. func (jm *jobManager) Submit(logger *zap.Logger, name string, job func() error) { jm.mu.Lock() defer jm.mu.Unlock() if jm.names == nil { jm.names = make(map[string]struct{}) } if name != "" { // prevent duplicate jobs if _, ok := jm.names[name]; ok { return } jm.names[name] = struct{}{} } jm.queue = append(jm.queue, namedJob{name, job, logger}) if jm.activeWorkers < jm.maxConcurrentJobs { jm.activeWorkers++ go jm.worker() } } func (jm *jobManager) worker() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: certificate worker: %v\n%s", err, buf) } }() for { jm.mu.Lock() if len(jm.queue) == 0 { jm.activeWorkers-- jm.mu.Unlock() return } next := jm.queue[0] jm.queue = jm.queue[1:] jm.mu.Unlock() if err := next.job(); err != nil { if next.logger != nil { next.logger.Error("job failed", zap.Error(err)) } } if next.name != "" { jm.mu.Lock() delete(jm.names, next.name) jm.mu.Unlock() } } } func doWithRetry(ctx context.Context, log *zap.Logger, f func(context.Context) error) error { var attempts int ctx = context.WithValue(ctx, AttemptsCtxKey, &attempts) // the initial intervalIndex is -1, signaling // that we should not wait for the first attempt start, intervalIndex := time.Now(), -1 var err error for time.Since(start) < maxRetryDuration { var wait time.Duration if intervalIndex >= 0 { wait = retryIntervals[intervalIndex] } timer := time.NewTimer(wait) select { case <-ctx.Done(): timer.Stop() return context.Canceled case <-timer.C: err = f(ctx) attempts++ if err == nil || errors.Is(err, context.Canceled) { return err } var errNoRetry ErrNoRetry if errors.As(err, &errNoRetry) { return err } if intervalIndex < len(retryIntervals)-1 { intervalIndex++ } if time.Since(start) < maxRetryDuration { if log != nil { log.Error("will retry", zap.Error(err), zap.Int("attempt", attempts), zap.Duration("retrying_in", retryIntervals[intervalIndex]), zap.Duration("elapsed", time.Since(start)), zap.Duration("max_duration", maxRetryDuration)) } } else { if log != nil { log.Error("final attempt; giving up", zap.Error(err), zap.Int("attempt", attempts), zap.Duration("elapsed", time.Since(start)), zap.Duration("max_duration", maxRetryDuration)) } return nil } } } return err } // ErrNoRetry is an error type which signals // to stop retries early. type ErrNoRetry struct{ Err error } // Unwrap makes it so that e wraps e.Err. func (e ErrNoRetry) Unwrap() error { return e.Err } func (e ErrNoRetry) Error() string { return e.Err.Error() } type retryStateCtxKey struct{} // AttemptsCtxKey is the context key for the value // that holds the attempt counter. The value counts // how many times the operation has been attempted. // A value of 0 means first attempt. var AttemptsCtxKey retryStateCtxKey // retryIntervals are based on the idea of exponential // backoff, but weighed a little more heavily to the // front. We figure that intermittent errors would be // resolved after the first retry, but any errors after // that would probably require at least a few minutes // to clear up: either for DNS to propagate, for the // administrator to fix their DNS or network properties, // or some other external factor needs to change. We // chose intervals that we think will be most useful // without introducing unnecessary delay. The last // interval in this list will be used until the time // of maxRetryDuration has elapsed. var retryIntervals = []time.Duration{ 1 * time.Minute, 2 * time.Minute, 2 * time.Minute, 5 * time.Minute, // elapsed: 10 min 10 * time.Minute, 20 * time.Minute, 20 * time.Minute, // elapsed: 1 hr 30 * time.Minute, 30 * time.Minute, // elapsed: 2 hr 1 * time.Hour, 3 * time.Hour, // elapsed: 6 hr 6 * time.Hour, // for up to maxRetryDuration } // maxRetryDuration is the maximum duration to try // doing retries using the above intervals. const maxRetryDuration = 24 * time.Hour * 30 golang-github-caddyserver-certmagic-0.15.2/cache.go000066400000000000000000000271711414454377300222110ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "fmt" weakrand "math/rand" // seeded elsewhere "strings" "sync" "time" "go.uber.org/zap" ) // Cache is a structure that stores certificates in memory. // A Cache indexes certificates by name for quick access // during TLS handshakes, and avoids duplicating certificates // in memory. Generally, there should only be one per process. // However, that is not a strict requirement; but using more // than one is a code smell, and may indicate an // over-engineered design. // // An empty cache is INVALID and must not be used. Be sure // to call NewCache to get a valid value. // // These should be very long-lived values and must not be // copied. Before all references leave scope to be garbage // collected, ensure you call Stop() to stop maintenance on // the certificates stored in this cache and release locks. // // Caches are not usually manipulated directly; create a // Config value with a pointer to a Cache, and then use // the Config to interact with the cache. Caches are // agnostic of any particular storage or ACME config, // since each certificate may be managed and stored // differently. type Cache struct { // User configuration of the cache options CacheOptions // The cache is keyed by certificate hash cache map[string]Certificate // cacheIndex is a map of SAN to cache key (cert hash) cacheIndex map[string][]string // Protects the cache and index maps mu sync.RWMutex // Close this channel to cancel asset maintenance stopChan chan struct{} // Used to signal when stopping is completed doneChan chan struct{} logger *zap.Logger } // NewCache returns a new, valid Cache for efficiently // accessing certificates in memory. It also begins a // maintenance goroutine to tend to the certificates // in the cache. Call Stop() when you are done with the // cache so it can clean up locks and stuff. // // Most users of this package will not need to call this // because a default certificate cache is created for you. // Only advanced use cases require creating a new cache. // // This function panics if opts.GetConfigForCert is not // set. The reason is that a cache absolutely needs to // be able to get a Config with which to manage TLS // assets, and it is not safe to assume that the Default // config is always the correct one, since you have // created the cache yourself. // // See the godoc for Cache to use it properly. When // no longer needed, caches should be stopped with // Stop() to clean up resources even if the process // is being terminated, so that it can clean up // any locks for other processes to unblock! func NewCache(opts CacheOptions) *Cache { // assume default options if necessary if opts.OCSPCheckInterval <= 0 { opts.OCSPCheckInterval = DefaultOCSPCheckInterval } if opts.RenewCheckInterval <= 0 { opts.RenewCheckInterval = DefaultRenewCheckInterval } if opts.Capacity < 0 { opts.Capacity = 0 } // this must be set, because we cannot not // safely assume that the Default Config // is always the correct one to use if opts.GetConfigForCert == nil { panic("cache must be initialized with a GetConfigForCert callback") } c := &Cache{ options: opts, cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), stopChan: make(chan struct{}), doneChan: make(chan struct{}), logger: opts.Logger, } go c.maintainAssets(0) return c } // Stop stops the maintenance goroutine for // certificates in certCache. It blocks until // stopping is complete. Once a cache is // stopped, it cannot be reused. func (certCache *Cache) Stop() { close(certCache.stopChan) // signal to stop <-certCache.doneChan // wait for stop to complete } // CacheOptions is used to configure certificate caches. // Once a cache has been created with certain options, // those settings cannot be changed. type CacheOptions struct { // REQUIRED. A function that returns a configuration // used for managing a certificate, or for accessing // that certificate's asset storage (e.g. for // OCSP staples, etc). The returned Config MUST // be associated with the same Cache as the caller. // // The reason this is a callback function, dynamically // returning a Config (instead of attaching a static // pointer to a Config on each certificate) is because // the config for how to manage a domain's certificate // might change from maintenance to maintenance. The // cache is so long-lived, we cannot assume that the // host's situation will always be the same; e.g. the // certificate might switch DNS providers, so the DNS // challenge (if used) would need to be adjusted from // the last time it was run ~8 weeks ago. GetConfigForCert ConfigGetter // How often to check certificates for renewal; // if unset, DefaultOCSPCheckInterval will be used. OCSPCheckInterval time.Duration // How often to check certificates for renewal; // if unset, DefaultRenewCheckInterval will be used. RenewCheckInterval time.Duration // Maximum number of certificates to allow in the cache. // If reached, certificates will be randomly evicted to // make room for new ones. 0 means unlimited. Capacity int // Set a logger to enable logging Logger *zap.Logger } // ConfigGetter is a function that returns a prepared, // valid config that should be used when managing the // given certificate or its assets. type ConfigGetter func(Certificate) (*Config, error) // cacheCertificate calls unsyncedCacheCertificate with a write lock. // // This function is safe for concurrent use. func (certCache *Cache) cacheCertificate(cert Certificate) { certCache.mu.Lock() certCache.unsyncedCacheCertificate(cert) certCache.mu.Unlock() } // unsyncedCacheCertificate adds cert to the in-memory cache unless // it already exists in the cache (according to cert.Hash). It // updates the name index. // // This function is NOT safe for concurrent use. Callers MUST acquire // a write lock on certCache.mu first. func (certCache *Cache) unsyncedCacheCertificate(cert Certificate) { // no-op if this certificate already exists in the cache if _, ok := certCache.cache[cert.hash]; ok { if certCache.logger != nil { certCache.logger.Debug("certificate already cached", zap.Strings("subjects", cert.Names), zap.Time("expiration", cert.Leaf.NotAfter), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash)) } return } // if the cache is at capacity, make room for new cert cacheSize := len(certCache.cache) if certCache.options.Capacity > 0 && cacheSize >= certCache.options.Capacity { // Go maps are "nondeterministic" but not actually random, // so although we could just chop off the "front" of the // map with less code, that is a heavily skewed eviction // strategy; generating random numbers is cheap and // ensures a much better distribution. rnd := weakrand.Intn(cacheSize) i := 0 for _, randomCert := range certCache.cache { if i == rnd { if certCache.logger != nil { certCache.logger.Debug("cache full; evicting random certificate", zap.Strings("removing_subjects", randomCert.Names), zap.String("removing_hash", randomCert.hash), zap.Strings("inserting_subjects", cert.Names), zap.String("inserting_hash", cert.hash)) } certCache.removeCertificate(randomCert) break } i++ } } // store the certificate certCache.cache[cert.hash] = cert // update the index so we can access it by name for _, name := range cert.Names { certCache.cacheIndex[name] = append(certCache.cacheIndex[name], cert.hash) } if certCache.logger != nil { certCache.logger.Debug("added certificate to cache", zap.Strings("subjects", cert.Names), zap.Time("expiration", cert.Leaf.NotAfter), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash), zap.Int("cache_size", len(certCache.cache)), zap.Int("cache_capacity", certCache.options.Capacity)) } } // removeCertificate removes cert from the cache. // // This function is NOT safe for concurrent use; callers // MUST first acquire a write lock on certCache.mu. func (certCache *Cache) removeCertificate(cert Certificate) { // delete all mentions of this cert from the name index for _, name := range cert.Names { keyList := certCache.cacheIndex[name] for i := 0; i < len(keyList); i++ { if keyList[i] == cert.hash { keyList = append(keyList[:i], keyList[i+1:]...) i-- } } if len(keyList) == 0 { delete(certCache.cacheIndex, name) } else { certCache.cacheIndex[name] = keyList } } // delete the actual cert from the cache delete(certCache.cache, cert.hash) if certCache.logger != nil { certCache.logger.Debug("removed certificate from cache", zap.Strings("subjects", cert.Names), zap.Time("expiration", cert.Leaf.NotAfter), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash), zap.Int("cache_size", len(certCache.cache)), zap.Int("cache_capacity", certCache.options.Capacity)) } } // replaceCertificate atomically replaces oldCert with newCert in // the cache. // // This method is safe for concurrent use. func (certCache *Cache) replaceCertificate(oldCert, newCert Certificate) { certCache.mu.Lock() certCache.removeCertificate(oldCert) certCache.unsyncedCacheCertificate(newCert) certCache.mu.Unlock() if certCache.logger != nil { certCache.logger.Info("replaced certificate in cache", zap.Strings("subjects", newCert.Names), zap.Time("new_expiration", newCert.Leaf.NotAfter)) } } func (certCache *Cache) getAllMatchingCerts(name string) []Certificate { certCache.mu.RLock() defer certCache.mu.RUnlock() allCertKeys := certCache.cacheIndex[name] certs := make([]Certificate, len(allCertKeys)) for i := range allCertKeys { certs[i] = certCache.cache[allCertKeys[i]] } return certs } func (certCache *Cache) getAllCerts() []Certificate { certCache.mu.RLock() defer certCache.mu.RUnlock() certs := make([]Certificate, 0, len(certCache.cache)) for _, cert := range certCache.cache { certs = append(certs, cert) } return certs } func (certCache *Cache) getConfig(cert Certificate) (*Config, error) { cfg, err := certCache.options.GetConfigForCert(cert) if err != nil { return nil, err } if cfg.certCache != nil && cfg.certCache != certCache { return nil, fmt.Errorf("config returned for certificate %v is not nil and points to different cache; got %p, expected %p (this one)", cert.Names, cfg.certCache, certCache) } return cfg, nil } // AllMatchingCertificates returns a list of all certificates that could // be used to serve the given SNI name, including exact SAN matches and // wildcard matches. func (certCache *Cache) AllMatchingCertificates(name string) []Certificate { // get exact matches first certs := certCache.getAllMatchingCerts(name) // then look for wildcard matches by replacing each // label of the domain name with wildcards labels := strings.Split(name, ".") for i := range labels { labels[i] = "*" candidate := strings.Join(labels, ".") certs = append(certs, certCache.getAllMatchingCerts(candidate)...) } return certs } var ( defaultCache *Cache defaultCacheMu sync.Mutex ) golang-github-caddyserver-certmagic-0.15.2/cache_test.go000066400000000000000000000026411414454377300232430ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import "testing" func TestNewCache(t *testing.T) { noop := func(Certificate) (*Config, error) { return new(Config), nil } c := NewCache(CacheOptions{GetConfigForCert: noop}) defer c.Stop() if c.options.RenewCheckInterval != DefaultRenewCheckInterval { t.Errorf("Expected RenewCheckInterval to be set to default value, but it wasn't: %s", c.options.RenewCheckInterval) } if c.options.OCSPCheckInterval != DefaultOCSPCheckInterval { t.Errorf("Expected OCSPCheckInterval to be set to default value, but it wasn't: %s", c.options.OCSPCheckInterval) } if c.options.GetConfigForCert == nil { t.Error("Expected GetConfigForCert to be set, but it was nil") } if c.cache == nil { t.Error("Expected cache to be set, but it was nil") } if c.stopChan == nil { t.Error("Expected stopChan to be set, but it was nil") } } golang-github-caddyserver-certmagic-0.15.2/certificates.go000066400000000000000000000335571414454377300236200ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto/tls" "crypto/x509" "fmt" "io/ioutil" "net" "strings" "time" "go.uber.org/zap" "golang.org/x/crypto/ocsp" ) // Certificate is a tls.Certificate with associated metadata tacked on. // Even if the metadata can be obtained by parsing the certificate, // we are more efficient by extracting the metadata onto this struct, // but at the cost of slightly higher memory use. type Certificate struct { tls.Certificate // Names is the list of subject names this // certificate is signed for. Names []string // Optional; user-provided, and arbitrary. Tags []string // OCSP contains the certificate's parsed OCSP response. ocsp *ocsp.Response // The hex-encoded hash of this cert's chain's bytes. hash string // Whether this certificate is under our management. managed bool // The unique string identifying the issuer of this certificate. issuerKey string } // NeedsRenewal returns true if the certificate is // expiring soon (according to cfg) or has expired. func (cert Certificate) NeedsRenewal(cfg *Config) bool { return currentlyInRenewalWindow(cert.Leaf.NotBefore, cert.Leaf.NotAfter, cfg.RenewalWindowRatio) } // Expired returns true if the certificate has expired. func (cert Certificate) Expired() bool { if cert.Leaf == nil { // ideally cert.Leaf would never be nil, but this can happen for // "synthetic" certs like those made to solve the TLS-ALPN challenge // which adds a special cert directly to the cache, since // tls.X509KeyPair() discards the leaf; oh well return false } return time.Now().After(cert.Leaf.NotAfter) } // currentlyInRenewalWindow returns true if the current time is // within the renewal window, according to the given start/end // dates and the ratio of the renewal window. If true is returned, // the certificate being considered is due for renewal. func currentlyInRenewalWindow(notBefore, notAfter time.Time, renewalWindowRatio float64) bool { if notAfter.IsZero() { return false } lifetime := notAfter.Sub(notBefore) if renewalWindowRatio == 0 { renewalWindowRatio = DefaultRenewalWindowRatio } renewalWindow := time.Duration(float64(lifetime) * renewalWindowRatio) renewalWindowStart := notAfter.Add(-renewalWindow) return time.Now().After(renewalWindowStart) } // HasTag returns true if cert.Tags has tag. func (cert Certificate) HasTag(tag string) bool { for _, t := range cert.Tags { if t == tag { return true } } return false } // CacheManagedCertificate loads the certificate for domain into the // cache, from the TLS storage for managed certificates. It returns a // copy of the Certificate that was put into the cache. // // This is a lower-level method; normally you'll call Manage() instead. // // This method is safe for concurrent use. func (cfg *Config) CacheManagedCertificate(domain string) (Certificate, error) { cert, err := cfg.loadManagedCertificate(domain) if err != nil { return cert, err } cfg.certCache.cacheCertificate(cert) cfg.emit("cached_managed_cert", cert.Names) return cert, nil } // loadManagedCertificate loads the managed certificate for domain from any // of the configured issuers' storage locations, but it does not add it to // the cache. It just loads from storage and returns it. func (cfg *Config) loadManagedCertificate(domain string) (Certificate, error) { certRes, err := cfg.loadCertResourceAnyIssuer(domain) if err != nil { return Certificate{}, err } cert, err := cfg.makeCertificateWithOCSP(certRes.CertificatePEM, certRes.PrivateKeyPEM) if err != nil { return cert, err } cert.managed = true cert.issuerKey = certRes.issuerKey return cert, nil } // CacheUnmanagedCertificatePEMFile loads a certificate for host using certFile // and keyFile, which must be in PEM format. It stores the certificate in // the in-memory cache. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedCertificatePEMFile(certFile, keyFile string, tags []string) error { cert, err := cfg.makeCertificateFromDiskWithOCSP(cfg.Storage, certFile, keyFile) if err != nil { return err } cert.Tags = tags cfg.certCache.cacheCertificate(cert) cfg.emit("cached_unmanaged_cert", cert.Names) return nil } // CacheUnmanagedTLSCertificate adds tlsCert to the certificate cache. // It staples OCSP if possible. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedTLSCertificate(tlsCert tls.Certificate, tags []string) error { var cert Certificate err := fillCertFromLeaf(&cert, tlsCert) if err != nil { return err } _, err = stapleOCSP(cfg.OCSP, cfg.Storage, &cert, nil) if err != nil && cfg.Logger != nil { cfg.Logger.Warn("stapling OCSP", zap.Error(err)) } cfg.emit("cached_unmanaged_cert", cert.Names) cert.Tags = tags cfg.certCache.cacheCertificate(cert) return nil } // CacheUnmanagedCertificatePEMBytes makes a certificate out of the PEM bytes // of the certificate and key, then caches it in memory. // // This method is safe for concurrent use. func (cfg *Config) CacheUnmanagedCertificatePEMBytes(certBytes, keyBytes []byte, tags []string) error { cert, err := cfg.makeCertificateWithOCSP(certBytes, keyBytes) if err != nil { return err } cert.Tags = tags cfg.certCache.cacheCertificate(cert) cfg.emit("cached_unmanaged_cert", cert.Names) return nil } // makeCertificateFromDiskWithOCSP makes a Certificate by loading the // certificate and key files. It fills out all the fields in // the certificate except for the Managed and OnDemand flags. // (It is up to the caller to set those.) It staples OCSP. func (cfg Config) makeCertificateFromDiskWithOCSP(storage Storage, certFile, keyFile string) (Certificate, error) { certPEMBlock, err := ioutil.ReadFile(certFile) if err != nil { return Certificate{}, err } keyPEMBlock, err := ioutil.ReadFile(keyFile) if err != nil { return Certificate{}, err } return cfg.makeCertificateWithOCSP(certPEMBlock, keyPEMBlock) } // makeCertificateWithOCSP is the same as makeCertificate except that it also // staples OCSP to the certificate. func (cfg Config) makeCertificateWithOCSP(certPEMBlock, keyPEMBlock []byte) (Certificate, error) { cert, err := makeCertificate(certPEMBlock, keyPEMBlock) if err != nil { return cert, err } _, err = stapleOCSP(cfg.OCSP, cfg.Storage, &cert, certPEMBlock) if err != nil && cfg.Logger != nil { cfg.Logger.Warn("stapling OCSP", zap.Error(err)) } return cert, nil } // makeCertificate turns a certificate PEM bundle and a key PEM block into // a Certificate with necessary metadata from parsing its bytes filled into // its struct fields for convenience (except for the OnDemand and Managed // flags; it is up to the caller to set those properties!). This function // does NOT staple OCSP. func makeCertificate(certPEMBlock, keyPEMBlock []byte) (Certificate, error) { var cert Certificate // Convert to a tls.Certificate tlsCert, err := tls.X509KeyPair(certPEMBlock, keyPEMBlock) if err != nil { return cert, err } // Extract necessary metadata err = fillCertFromLeaf(&cert, tlsCert) if err != nil { return cert, err } return cert, nil } // fillCertFromLeaf populates cert from tlsCert. If it succeeds, it // guarantees that cert.Leaf is non-nil. func fillCertFromLeaf(cert *Certificate, tlsCert tls.Certificate) error { if len(tlsCert.Certificate) == 0 { return fmt.Errorf("certificate is empty") } cert.Certificate = tlsCert // the leaf cert should be the one for the site; we must set // the tls.Certificate.Leaf field so that TLS handshakes are // more efficient leaf, err := x509.ParseCertificate(tlsCert.Certificate[0]) if err != nil { return err } cert.Certificate.Leaf = leaf // for convenience, we do want to assemble all the // subjects on the certificate into one list if leaf.Subject.CommonName != "" { // TODO: CommonName is deprecated cert.Names = []string{strings.ToLower(leaf.Subject.CommonName)} } for _, name := range leaf.DNSNames { if name != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(name)) } } for _, ip := range leaf.IPAddresses { if ipStr := ip.String(); ipStr != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(ipStr)) } } for _, email := range leaf.EmailAddresses { if email != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, strings.ToLower(email)) } } for _, u := range leaf.URIs { if u.String() != leaf.Subject.CommonName { // TODO: CommonName is deprecated cert.Names = append(cert.Names, u.String()) } } if len(cert.Names) == 0 { return fmt.Errorf("certificate has no names") } cert.hash = hashCertificateChain(cert.Certificate.Certificate) return nil } // managedCertInStorageExpiresSoon returns true if cert (being a // managed certificate) is expiring within RenewDurationBefore. // It returns false if there was an error checking the expiration // of the certificate as found in storage, or if the certificate // in storage is NOT expiring soon. A certificate that is expiring // soon in our cache but is not expiring soon in storage probably // means that another instance renewed the certificate in the // meantime, and it would be a good idea to simply load the cert // into our cache rather than repeating the renewal process again. func (cfg *Config) managedCertInStorageExpiresSoon(cert Certificate) (bool, error) { certRes, err := cfg.loadCertResourceAnyIssuer(cert.Names[0]) if err != nil { return false, err } _, needsRenew := cfg.managedCertNeedsRenewal(certRes) return needsRenew, nil } // reloadManagedCertificate reloads the certificate corresponding to the name(s) // on oldCert into the cache, from storage. This also replaces the old certificate // with the new one, so that all configurations that used the old cert now point // to the new cert. It assumes that the new certificate for oldCert.Names[0] is // already in storage. func (cfg *Config) reloadManagedCertificate(oldCert Certificate) error { if cfg.Logger != nil { cfg.Logger.Info("reloading managed certificate", zap.Strings("identifiers", oldCert.Names)) } newCert, err := cfg.loadManagedCertificate(oldCert.Names[0]) if err != nil { return fmt.Errorf("loading managed certificate for %v from storage: %v", oldCert.Names, err) } cfg.certCache.replaceCertificate(oldCert, newCert) return nil } // SubjectQualifiesForCert returns true if subj is a name which, // as a quick sanity check, looks like it could be the subject // of a certificate. Requirements are: // - must not be empty // - must not start or end with a dot (RFC 1034) // - must not contain common accidental special characters func SubjectQualifiesForCert(subj string) bool { // must not be empty return strings.TrimSpace(subj) != "" && // must not start or end with a dot !strings.HasPrefix(subj, ".") && !strings.HasSuffix(subj, ".") && // if it has a wildcard, must be a left-most label (or exactly "*" // which won't be trusted by browsers but still technically works) (!strings.Contains(subj, "*") || strings.HasPrefix(subj, "*.") || subj == "*") && // must not contain other common special characters !strings.ContainsAny(subj, "()[]{}<> \t\n\"\\!@#$%^&|;'+=") } // SubjectQualifiesForPublicCert returns true if the subject // name appears eligible for automagic TLS with a public // CA such as Let's Encrypt. For example: localhost and IP // addresses are not eligible because we cannot obtain certs // for those names with a public CA. Wildcard names are // allowed, as long as they conform to CABF requirements (only // one wildcard label, and it must be the left-most label). func SubjectQualifiesForPublicCert(subj string) bool { // must at least qualify for a certificate return SubjectQualifiesForCert(subj) && // localhost, .localhost TLD, and .local TLD are ineligible !SubjectIsInternal(subj) && // cannot be an IP address (as of yet), see // https://community.letsencrypt.org/t/certificate-for-static-ip/84/2?u=mholt !SubjectIsIP(subj) && // only one wildcard label allowed, and it must be left-most, with 3+ labels (!strings.Contains(subj, "*") || (strings.Count(subj, "*") == 1 && strings.Count(subj, ".") > 1 && len(subj) > 2 && strings.HasPrefix(subj, "*."))) } // SubjectIsIP returns true if subj is an IP address. func SubjectIsIP(subj string) bool { return net.ParseIP(subj) != nil } // SubjectIsInternal returns true if subj is an internal-facing // hostname or address. func SubjectIsInternal(subj string) bool { return subj == "localhost" || strings.HasSuffix(subj, ".localhost") || strings.HasSuffix(subj, ".local") } // MatchWildcard returns true if subject (a candidate DNS name) // matches wildcard (a reference DNS name), mostly according to // RFC 6125-compliant wildcard rules. See also RFC 2818 which // states that IP addresses must match exactly, but this function // does not attempt to distinguish IP addresses from internal or // external DNS names that happen to look like IP addresses. // It uses DNS wildcard matching logic. // https://tools.ietf.org/html/rfc2818#section-3.1 func MatchWildcard(subject, wildcard string) bool { if subject == wildcard { return true } if !strings.Contains(wildcard, "*") { return false } labels := strings.Split(subject, ".") for i := range labels { if labels[i] == "" { continue // invalid label } labels[i] = "*" candidate := strings.Join(labels, ".") if candidate == wildcard { return true } } return false } golang-github-caddyserver-certmagic-0.15.2/certificates_test.go000066400000000000000000000167401414454377300246520ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto/tls" "crypto/x509" "reflect" "testing" "time" ) func TestUnexportedGetCertificate(t *testing.T) { certCache := &Cache{cache: make(map[string]Certificate), cacheIndex: make(map[string][]string)} cfg := &Config{certCache: certCache} // When cache is empty if _, matched, defaulted := cfg.getCertificate(&tls.ClientHelloInfo{ServerName: "example.com"}); matched || defaulted { t.Errorf("Got a certificate when cache was empty; matched=%v, defaulted=%v", matched, defaulted) } // When cache has one certificate in it firstCert := Certificate{Names: []string{"example.com"}} certCache.cache["0xdeadbeef"] = firstCert certCache.cacheIndex["example.com"] = []string{"0xdeadbeef"} if cert, matched, defaulted := cfg.getCertificate(&tls.ClientHelloInfo{ServerName: "example.com"}); !matched || defaulted || cert.Names[0] != "example.com" { t.Errorf("Didn't get a cert for 'example.com' or got the wrong one: %v, matched=%v, defaulted=%v", cert, matched, defaulted) } // When retrieving wildcard certificate certCache.cache["0xb01dface"] = Certificate{Names: []string{"*.example.com"}} certCache.cacheIndex["*.example.com"] = []string{"0xb01dface"} if cert, matched, defaulted := cfg.getCertificate(&tls.ClientHelloInfo{ServerName: "sub.example.com"}); !matched || defaulted || cert.Names[0] != "*.example.com" { t.Errorf("Didn't get wildcard cert for 'sub.example.com' or got the wrong one: %v, matched=%v, defaulted=%v", cert, matched, defaulted) } // When no certificate matches and SNI is provided, return no certificate (should be TLS alert) if cert, matched, defaulted := cfg.getCertificate(&tls.ClientHelloInfo{ServerName: "nomatch"}); matched || defaulted { t.Errorf("Expected matched=false, defaulted=false; but got matched=%v, defaulted=%v (cert: %v)", matched, defaulted, cert) } } func TestCacheCertificate(t *testing.T) { certCache := &Cache{cache: make(map[string]Certificate), cacheIndex: make(map[string][]string)} certCache.cacheCertificate(Certificate{Names: []string{"example.com", "sub.example.com"}, hash: "foobar", Certificate: tls.Certificate{Leaf: &x509.Certificate{NotAfter: time.Now()}}}) if len(certCache.cache) != 1 { t.Errorf("Expected length of certificate cache to be 1") } if _, ok := certCache.cache["foobar"]; !ok { t.Error("Expected first cert to be cached by key 'foobar', but it wasn't") } if _, ok := certCache.cacheIndex["example.com"]; !ok { t.Error("Expected first cert to be keyed by 'example.com', but it wasn't") } if _, ok := certCache.cacheIndex["sub.example.com"]; !ok { t.Error("Expected first cert to be keyed by 'sub.example.com', but it wasn't") } // using same cache; and has cert with overlapping name, but different hash certCache.cacheCertificate(Certificate{Names: []string{"example.com"}, hash: "barbaz", Certificate: tls.Certificate{Leaf: &x509.Certificate{NotAfter: time.Now()}}}) if _, ok := certCache.cache["barbaz"]; !ok { t.Error("Expected second cert to be cached by key 'barbaz.com', but it wasn't") } if hashes, ok := certCache.cacheIndex["example.com"]; !ok { t.Error("Expected second cert to be keyed by 'example.com', but it wasn't") } else if !reflect.DeepEqual(hashes, []string{"foobar", "barbaz"}) { t.Errorf("Expected second cert to map to 'barbaz' but it was %v instead", hashes) } } func TestSubjectQualifiesForCert(t *testing.T) { for i, test := range []struct { host string expect bool }{ {"hostname", true}, {"example.com", true}, {"sub.example.com", true}, {"Sub.Example.COM", true}, {"127.0.0.1", true}, {"127.0.1.5", true}, {"69.123.43.94", true}, {"::1", true}, {"::", true}, {"0.0.0.0", true}, {"", false}, {" ", false}, {"*.example.com", true}, {"*.*.example.com", true}, {"sub.*.example.com", false}, {"*sub.example.com", false}, {"**.tld", false}, {"*", true}, {"*.tld", true}, {".tld", false}, {"example.com.", false}, {"localhost", true}, {"foo.localhost", true}, {"local", true}, {"192.168.1.3", true}, {"10.0.2.1", true}, {"169.112.53.4", true}, {"$hostname", false}, {"%HOSTNAME%", false}, {"{hostname}", false}, {"hostname!", false}, {"", false}, {"# hostname", false}, {"// hostname", false}, {"user@hostname", false}, {"hostname;", false}, {`"hostname"`, false}, } { actual := SubjectQualifiesForCert(test.host) if actual != test.expect { t.Errorf("Test %d: Expected SubjectQualifiesForCert(%s)=%v, but got %v", i, test.host, test.expect, actual) } } } func TestSubjectQualifiesForPublicCert(t *testing.T) { for i, test := range []struct { host string expect bool }{ {"hostname", true}, {"example.com", true}, {"sub.example.com", true}, {"Sub.Example.COM", true}, {"127.0.0.1", false}, {"127.0.1.5", false}, {"69.123.43.94", false}, {"::1", false}, {"::", false}, {"0.0.0.0", false}, {"", false}, {" ", false}, {"*.example.com", true}, {"*.*.example.com", false}, {"sub.*.example.com", false}, {"*sub.example.com", false}, {"*", false}, // won't be trusted by browsers {"*.tld", false}, // won't be trusted by browsers {".tld", false}, {"example.com.", false}, {"localhost", false}, {"foo.localhost", false}, {"local", true}, {"foo.local", false}, {"foo.bar.local", false}, {"192.168.1.3", false}, {"10.0.2.1", false}, {"169.112.53.4", false}, {"$hostname", false}, {"%HOSTNAME%", false}, {"{hostname}", false}, {"hostname!", false}, {"", false}, {"# hostname", false}, {"// hostname", false}, {"user@hostname", false}, {"hostname;", false}, {`"hostname"`, false}, } { actual := SubjectQualifiesForPublicCert(test.host) if actual != test.expect { t.Errorf("Test %d: Expected SubjectQualifiesForPublicCert(%s)=%v, but got %v", i, test.host, test.expect, actual) } } } func TestMatchWildcard(t *testing.T) { for i, test := range []struct { subject, wildcard string expect bool }{ {"hostname", "hostname", true}, {"foo.localhost", "foo.localhost", true}, {"foo.localhost", "bar.localhost", false}, {"foo.localhost", "*.localhost", true}, {"bar.localhost", "*.localhost", true}, {"foo.bar.localhost", "*.localhost", false}, {".localhost", "*.localhost", false}, {"foo.localhost", "foo.*", false}, {"foo.bar.local", "foo.*.local", false}, {"foo.bar.local", "foo.bar.*", false}, {"foo.bar.local", "*.bar.local", true}, {"1.2.3.4.5.6", "*.2.3.4.5.6", true}, {"1.2.3.4.5.6", "*.*.3.4.5.6", true}, {"1.2.3.4.5.6", "*.*.*.4.5.6", true}, {"1.2.3.4.5.6", "*.*.*.*.5.6", true}, {"1.2.3.4.5.6", "*.*.*.*.*.6", true}, {"1.2.3.4.5.6", "*.*.*.*.*.*", true}, {"0.1.2.3.4.5.6", "*.*.*.*.*.*", false}, {"1.2.3.4", "1.2.3.*", false}, // https://tools.ietf.org/html/rfc2818#section-3.1 } { actual := MatchWildcard(test.subject, test.wildcard) if actual != test.expect { t.Errorf("Test %d: Expected MatchWildcard(%s, %s)=%v, but got %v", i, test.subject, test.wildcard, test.expect, actual) } } } golang-github-caddyserver-certmagic-0.15.2/certmagic.go000066400000000000000000000405751414454377300231070ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. // Package certmagic automates the obtaining and renewal of TLS certificates, // including TLS & HTTPS best practices such as robust OCSP stapling, caching, // HTTP->HTTPS redirects, and more. // // Its high-level API serves your HTTP handlers over HTTPS if you simply give // the domain name(s) and the http.Handler; CertMagic will create and run // the HTTPS server for you, fully managing certificates during the lifetime // of the server. Similarly, it can be used to start TLS listeners or return // a ready-to-use tls.Config -- whatever layer you need TLS for, CertMagic // makes it easy. See the HTTPS, Listen, and TLS functions for that. // // If you need more control, create a Cache using NewCache() and then make // a Config using New(). You can then call Manage() on the config. But if // you use this lower-level API, you'll have to be sure to solve the HTTP // and TLS-ALPN challenges yourself (unless you disabled them or use the // DNS challenge) by using the provided Config.GetCertificate function // in your tls.Config and/or Config.HTTPChallangeHandler in your HTTP // handler. // // See the package's README for more instruction. package certmagic import ( "context" "crypto" "crypto/tls" "crypto/x509" "fmt" "log" "net" "net/http" "sort" "strings" "sync" "time" ) // HTTPS serves mux for all domainNames using the HTTP // and HTTPS ports, redirecting all HTTP requests to HTTPS. // It uses the Default config. // // This high-level convenience function is opinionated and // applies sane defaults for production use, including // timeouts for HTTP requests and responses. To allow very // long-lived connections, you should make your own // http.Server values and use this package's Listen(), TLS(), // or Config.TLSConfig() functions to customize to your needs. // For example, servers which need to support large uploads or // downloads with slow clients may need to use longer timeouts, // thus this function is not suitable. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func HTTPS(domainNames []string, mux http.Handler) error { if mux == nil { mux = http.DefaultServeMux } DefaultACME.Agreed = true cfg := NewDefault() err := cfg.ManageSync(context.Background(), domainNames) if err != nil { return err } httpWg.Add(1) defer httpWg.Done() // if we haven't made listeners yet, do so now, // and clean them up when all servers are done lnMu.Lock() if httpLn == nil && httpsLn == nil { httpLn, err = net.Listen("tcp", fmt.Sprintf(":%d", HTTPPort)) if err != nil { lnMu.Unlock() return err } tlsConfig := cfg.TLSConfig() tlsConfig.NextProtos = append([]string{"h2", "http/1.1"}, tlsConfig.NextProtos...) httpsLn, err = tls.Listen("tcp", fmt.Sprintf(":%d", HTTPSPort), tlsConfig) if err != nil { httpLn.Close() httpLn = nil lnMu.Unlock() return err } go func() { httpWg.Wait() lnMu.Lock() httpLn.Close() httpsLn.Close() lnMu.Unlock() }() } hln, hsln := httpLn, httpsLn lnMu.Unlock() // create HTTP/S servers that are configured // with sane default timeouts and appropriate // handlers (the HTTP server solves the HTTP // challenge and issues redirects to HTTPS, // while the HTTPS server simply serves the // user's handler) httpServer := &http.Server{ ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second, IdleTimeout: 5 * time.Second, } if len(cfg.Issuers) > 0 { if am, ok := cfg.Issuers[0].(*ACMEManager); ok { httpServer.Handler = am.HTTPChallengeHandler(http.HandlerFunc(httpRedirectHandler)) } } httpsServer := &http.Server{ ReadHeaderTimeout: 10 * time.Second, ReadTimeout: 30 * time.Second, WriteTimeout: 2 * time.Minute, IdleTimeout: 5 * time.Minute, Handler: mux, } log.Printf("%v Serving HTTP->HTTPS on %s and %s", domainNames, hln.Addr(), hsln.Addr()) go httpServer.Serve(hln) return httpsServer.Serve(hsln) } func httpRedirectHandler(w http.ResponseWriter, r *http.Request) { toURL := "https://" // since we redirect to the standard HTTPS port, we // do not need to include it in the redirect URL requestHost := hostOnly(r.Host) toURL += requestHost toURL += r.URL.RequestURI() // get rid of this disgusting unencrypted HTTP connection 🤢 w.Header().Set("Connection", "close") http.Redirect(w, r, toURL, http.StatusMovedPermanently) } // TLS enables management of certificates for domainNames // and returns a valid tls.Config. It uses the Default // config. // // Because this is a convenience function that returns // only a tls.Config, it does not assume HTTP is being // served on the HTTP port, so the HTTP challenge is // disabled (no HTTPChallengeHandler is necessary). The // package variable Default is modified so that the // HTTP challenge is disabled. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func TLS(domainNames []string) (*tls.Config, error) { DefaultACME.Agreed = true DefaultACME.DisableHTTPChallenge = true cfg := NewDefault() return cfg.TLSConfig(), cfg.ManageSync(context.Background(), domainNames) } // Listen manages certificates for domainName and returns a // TLS listener. It uses the Default config. // // Because this convenience function returns only a TLS-enabled // listener and does not presume HTTP is also being served, // the HTTP challenge will be disabled. The package variable // Default is modified so that the HTTP challenge is disabled. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func Listen(domainNames []string) (net.Listener, error) { DefaultACME.Agreed = true DefaultACME.DisableHTTPChallenge = true cfg := NewDefault() err := cfg.ManageSync(context.Background(), domainNames) if err != nil { return nil, err } return tls.Listen("tcp", fmt.Sprintf(":%d", HTTPSPort), cfg.TLSConfig()) } // ManageSync obtains certificates for domainNames and keeps them // renewed using the Default config. // // This is a slightly lower-level function; you will need to // wire up support for the ACME challenges yourself. You can // obtain a Config to help you do that by calling NewDefault(). // // You will need to ensure that you use a TLS config that gets // certificates from this Config and that the HTTP and TLS-ALPN // challenges can be solved. The easiest way to do this is to // use NewDefault().TLSConfig() as your TLS config and to wrap // your HTTP handler with NewDefault().HTTPChallengeHandler(). // If you don't have an HTTP server, you will need to disable // the HTTP challenge. // // If you already have a TLS config you want to use, you can // simply set its GetCertificate field to // NewDefault().GetCertificate. // // Calling this function signifies your acceptance to // the CA's Subscriber Agreement and/or Terms of Service. func ManageSync(ctx context.Context, domainNames []string) error { DefaultACME.Agreed = true return NewDefault().ManageSync(ctx, domainNames) } // ManageAsync is the same as ManageSync, except that // certificates are managed asynchronously. This means // that the function will return before certificates // are ready, and errors that occur during certificate // obtain or renew operations are only logged. It is // vital that you monitor the logs if using this method, // which is only recommended for automated/non-interactive // environments. func ManageAsync(ctx context.Context, domainNames []string) error { DefaultACME.Agreed = true return NewDefault().ManageAsync(ctx, domainNames) } // OnDemandConfig configures on-demand TLS (certificate // operations as-needed, like during TLS handshakes, // rather than immediately). // // When this package's high-level convenience functions // are used (HTTPS, Manage, etc., where the Default // config is used as a template), this struct regulates // certificate operations using an implicit whitelist // containing the names passed into those functions if // no DecisionFunc is set. This ensures some degree of // control by default to avoid certificate operations for // aribtrary domain names. To override this whitelist, // manually specify a DecisionFunc. To impose rate limits, // specify your own DecisionFunc. type OnDemandConfig struct { // If set, this function will be called to determine // whether a certificate can be obtained or renewed // for the given name. If an error is returned, the // request will be denied. DecisionFunc func(name string) error // List of whitelisted hostnames (SNI values) for // deferred (on-demand) obtaining of certificates. // Used only by higher-level functions in this // package to persist the list of hostnames that // the config is supposed to manage. This is done // because it seems reasonable that if you say // "Manage [domain names...]", then only those // domain names should be able to have certs; // we don't NEED this feature, but it makes sense // for higher-level convenience functions to be // able to retain their convenience (alternative // is: the user manually creates a DecisionFunc // that whitelists the same names it already // passed into Manage) and without letting clients // have their run of any domain names they want. // Only enforced if len > 0. hostWhitelist []string } func (o *OnDemandConfig) whitelistContains(name string) bool { for _, n := range o.hostWhitelist { if strings.EqualFold(n, name) { return true } } return false } // isLoopback returns true if the hostname of addr looks // explicitly like a common local hostname. addr must only // be a host or a host:port combination. func isLoopback(addr string) bool { host := hostOnly(addr) return host == "localhost" || strings.Trim(host, "[]") == "::1" || strings.HasPrefix(host, "127.") } // isInternal returns true if the IP of addr // belongs to a private network IP range. addr // must only be an IP or an IP:port combination. // Loopback addresses are considered false. func isInternal(addr string) bool { privateNetworks := []string{ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7", } host := hostOnly(addr) ip := net.ParseIP(host) if ip == nil { return false } for _, privateNetwork := range privateNetworks { _, ipnet, _ := net.ParseCIDR(privateNetwork) if ipnet.Contains(ip) { return true } } return false } // hostOnly returns only the host portion of hostport. // If there is no port or if there is an error splitting // the port off, the whole input string is returned. func hostOnly(hostport string) string { host, _, err := net.SplitHostPort(hostport) if err != nil { return hostport // OK; probably had no port to begin with } return host } // PreChecker is an interface that can be optionally implemented by // Issuers. Pre-checks are performed before each call (or batch of // identical calls) to Issue(), giving the issuer the option to ensure // it has all the necessary information/state. type PreChecker interface { PreCheck(ctx context.Context, names []string, interactive bool) error } // Issuer is a type that can issue certificates. type Issuer interface { // Issue obtains a certificate for the given CSR. It // must honor context cancellation if it is long-running. // It can also use the context to find out if the current // call is part of a retry, via AttemptsCtxKey. Issue(ctx context.Context, request *x509.CertificateRequest) (*IssuedCertificate, error) // IssuerKey must return a string that uniquely identifies // this particular configuration of the Issuer such that // any certificates obtained by this Issuer will be treated // as identical if they have the same SANs. // // Certificates obtained from Issuers with the same IssuerKey // will overwrite others with the same SANs. For example, an // Issuer might be able to obtain certificates from different // CAs, say A and B. It is likely that the CAs have different // use cases and purposes (e.g. testing and production), so // their respective certificates should not overwrite eaach // other. IssuerKey() string } // Revoker can revoke certificates. Reason codes are defined // by RFC 5280 §5.3.1: https://tools.ietf.org/html/rfc5280#section-5.3.1 // and are available as constants in our ACME library. type Revoker interface { Revoke(ctx context.Context, cert CertificateResource, reason int) error } // KeyGenerator can generate a private key. type KeyGenerator interface { // GenerateKey generates a private key. The returned // PrivateKey must be able to expose its associated // public key. GenerateKey() (crypto.PrivateKey, error) } // IssuedCertificate represents a certificate that was just issued. type IssuedCertificate struct { // The PEM-encoding of DER-encoded ASN.1 data. Certificate []byte // Any extra information to serialize alongside the // certificate in storage. Metadata interface{} } // CertificateResource associates a certificate with its private // key and other useful information, for use in maintaining the // certificate. type CertificateResource struct { // The list of names on the certificate; // for convenience only. SANs []string `json:"sans,omitempty"` // The PEM-encoding of DER-encoded ASN.1 data // for the cert or chain. CertificatePEM []byte `json:"-"` // The PEM-encoding of the certificate's private key. PrivateKeyPEM []byte `json:"-"` // Any extra information associated with the certificate, // usually provided by the issuer implementation. IssuerData interface{} `json:"issuer_data,omitempty"` // The unique string identifying the issuer of the // certificate; internally useful for storage access. issuerKey string `json:"-"` } // NamesKey returns the list of SANs as a single string, // truncated to some ridiculously long size limit. It // can act as a key for the set of names on the resource. func (cr *CertificateResource) NamesKey() string { sort.Strings(cr.SANs) result := strings.Join(cr.SANs, ",") if len(result) > 1024 { const trunc = "_trunc" result = result[:1024-len(trunc)] + trunc } return result } // Default contains the package defaults for the // various Config fields. This is used as a template // when creating your own Configs with New() or // NewDefault(), and it is also used as the Config // by all the high-level functions in this package // that abstract away most configuration (HTTPS(), // TLS(), Listen(), etc). // // The fields of this value will be used for Config // fields which are unset. Feel free to modify these // defaults, but do not use this Config by itself: it // is only a template. Valid configurations can be // obtained by calling New() (if you have your own // certificate cache) or NewDefault() (if you only // need a single config and want to use the default // cache). // // Even if the Issuers or Storage fields are not set, // defaults will be applied in the call to New(). var Default = Config{ RenewalWindowRatio: DefaultRenewalWindowRatio, Storage: defaultFileStorage, KeySource: DefaultKeyGenerator, } const ( // HTTPChallengePort is the officially-designated port for // the HTTP challenge according to the ACME spec. HTTPChallengePort = 80 // TLSALPNChallengePort is the officially-designated port for // the TLS-ALPN challenge according to the ACME spec. TLSALPNChallengePort = 443 ) // Port variables must remain their defaults unless you // forward packets from the defaults to whatever these // are set to; otherwise ACME challenges will fail. var ( // HTTPPort is the port on which to serve HTTP // and, as such, the HTTP challenge (unless // Default.AltHTTPPort is set). HTTPPort = 80 // HTTPSPort is the port on which to serve HTTPS // and, as such, the TLS-ALPN challenge // (unless Default.AltTLSALPNPort is set). HTTPSPort = 443 ) // Variables for conveniently serving HTTPS. var ( httpLn, httpsLn net.Listener lnMu sync.Mutex httpWg sync.WaitGroup ) // Maximum size for the stack trace when recovering from panics. const stackTraceBufferSize = 1024 * 128 golang-github-caddyserver-certmagic-0.15.2/certmagic_test.go000066400000000000000000000011511414454377300241310ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic // TODO golang-github-caddyserver-certmagic-0.15.2/config.go000066400000000000000000001073001414454377300224040ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "context" "crypto" "crypto/rand" "crypto/tls" "crypto/x509" "crypto/x509/pkix" "encoding/asn1" "encoding/json" "errors" "fmt" weakrand "math/rand" "net" "net/url" "strings" "time" "github.com/mholt/acmez" "github.com/mholt/acmez/acme" "go.uber.org/zap" "golang.org/x/net/idna" ) // Config configures a certificate manager instance. // An empty Config is not valid: use New() to obtain // a valid Config. type Config struct { // How much of a certificate's lifetime becomes the // renewal window, which is the span of time at the // end of the certificate's validity period in which // it should be renewed; for most certificates, the // global default is good, but for extremely short- // lived certs, you may want to raise this to ~0.5. RenewalWindowRatio float64 // An optional event callback clients can set // to subscribe to certain things happening // internally by this config; invocations are // synchronous, so make them return quickly! OnEvent func(event string, data interface{}) // DefaultServerName specifies a server name // to use when choosing a certificate if the // ClientHello's ServerName field is empty. DefaultServerName string // The state needed to operate on-demand TLS; // if non-nil, on-demand TLS is enabled and // certificate operations are deferred to // TLS handshakes (or as-needed). // TODO: Can we call this feature "Reactive/Lazy/Passive TLS" instead? OnDemand *OnDemandConfig // Adds the must staple TLS extension to the CSR. MustStaple bool // The source for getting new certificates; the // default Issuer is ACMEManager. If multiple // issuers are specified, they will be tried in // turn until one succeeds. Issuers []Issuer // The source of new private keys for certificates; // the default KeySource is StandardKeyGenerator. KeySource KeyGenerator // CertSelection chooses one of the certificates // with which the ClientHello will be completed; // if not set, DefaultCertificateSelector will // be used. CertSelection CertificateSelector // OCSP configures how OCSP is handled. By default, // OCSP responses are fetched for every certificate // with a responder URL, and cached on disk. Changing // these defaults is STRONGLY discouraged unless you // have a compelling reason to put clients at greater // risk and reduce their privacy. OCSP OCSPConfig // The storage to access when storing or loading // TLS assets. Default is the local file system. Storage Storage // Set a logger to enable logging. Logger *zap.Logger // required pointer to the in-memory cert cache certCache *Cache } // NewDefault makes a valid config based on the package // Default config. Most users will call this function // instead of New() since most use cases require only a // single config for any and all certificates. // // If your requirements are more advanced (for example, // multiple configs depending on the certificate), then use // New() instead. (You will need to make your own Cache // first.) If you only need a single Config to manage your // certs (even if that config changes, as long as it is the // only one), customize the Default package variable before // calling NewDefault(). // // All calls to NewDefault() will return configs that use the // same, default certificate cache. All configs returned // by NewDefault() are based on the values of the fields of // Default at the time it is called. // // This is the only way to get a config that uses the // default certificate cache. func NewDefault() *Config { defaultCacheMu.Lock() if defaultCache == nil { defaultCache = NewCache(CacheOptions{ // the cache will likely need to renew certificates, // so it will need to know how to do that, which // depends on the certificate being managed and which // can change during the lifetime of the cache; this // callback makes it possible to get the latest and // correct config with which to manage the cert, // but if the user does not provide one, we can only // assume that we are to use the default config GetConfigForCert: func(Certificate) (*Config, error) { return NewDefault(), nil }, }) } certCache := defaultCache defaultCacheMu.Unlock() return newWithCache(certCache, Default) } // New makes a new, valid config based on cfg and // uses the provided certificate cache. certCache // MUST NOT be nil or this function will panic. // // Use this method when you have an advanced use case // that requires a custom certificate cache and config // that may differ from the Default. For example, if // not all certificates are managed/renewed the same // way, you need to make your own Cache value with a // GetConfigForCert callback that returns the correct // configuration for each certificate. However, for // the vast majority of cases, there will be only a // single Config, thus the default cache (which always // uses the default Config) and default config will // suffice, and you should use NewDefault() instead. func New(certCache *Cache, cfg Config) *Config { if certCache == nil { panic("a certificate cache is required") } if certCache.options.GetConfigForCert == nil { panic("cache must have GetConfigForCert set in its options") } return newWithCache(certCache, cfg) } // newWithCache ensures that cfg is a valid config by populating // zero-value fields from the Default Config. If certCache is // nil, this function panics. func newWithCache(certCache *Cache, cfg Config) *Config { if certCache == nil { panic("cannot make a valid config without a pointer to a certificate cache") } if cfg.OnDemand == nil { cfg.OnDemand = Default.OnDemand } if cfg.RenewalWindowRatio == 0 { cfg.RenewalWindowRatio = Default.RenewalWindowRatio } if cfg.OnEvent == nil { cfg.OnEvent = Default.OnEvent } if cfg.KeySource == nil { cfg.KeySource = Default.KeySource } if cfg.DefaultServerName == "" { cfg.DefaultServerName = Default.DefaultServerName } if cfg.OnDemand == nil { cfg.OnDemand = Default.OnDemand } if !cfg.MustStaple { cfg.MustStaple = Default.MustStaple } if cfg.Storage == nil { cfg.Storage = Default.Storage } if len(cfg.Issuers) == 0 { cfg.Issuers = Default.Issuers if len(cfg.Issuers) == 0 { // at least one issuer is absolutely required cfg.Issuers = []Issuer{NewACMEManager(&cfg, DefaultACME)} } } // absolutely don't allow a nil storage, // because that would make almost anything // a config can do pointless if cfg.Storage == nil { cfg.Storage = defaultFileStorage } cfg.certCache = certCache return &cfg } // ManageSync causes the certificates for domainNames to be managed // according to cfg. If cfg.OnDemand is not nil, then this simply // whitelists the domain names and defers the certificate operations // to when they are needed. Otherwise, the certificates for each // name are loaded from storage or obtained from the CA. If loaded // from storage, they are renewed if they are expiring or expired. // It then caches the certificate in memory and is prepared to serve // them up during TLS handshakes. // // Note that name whitelisting for on-demand management only takes // effect if cfg.OnDemand.DecisionFunc is not set (is nil); it will // not overwrite an existing DecisionFunc, nor will it overwrite // its decision; i.e. the implicit whitelist is only used if no // DecisionFunc is set. // // This method is synchronous, meaning that certificates for all // domainNames must be successfully obtained (or renewed) before // it returns. It returns immediately on the first error for any // of the given domainNames. This behavior is recommended for // interactive use (i.e. when an administrator is present) so // that errors can be reported and fixed immediately. func (cfg *Config) ManageSync(ctx context.Context, domainNames []string) error { return cfg.manageAll(ctx, domainNames, false) } // ManageAsync is the same as ManageSync, except that ACME // operations are performed asynchronously (in the background). // This method returns before certificates are ready. It is // crucial that the administrator monitors the logs and is // notified of any errors so that corrective action can be // taken as soon as possible. Any errors returned from this // method occurred before ACME transactions started. // // As long as logs are monitored, this method is typically // recommended for non-interactive environments. // // If there are failures loading, obtaining, or renewing a // certificate, it will be retried with exponential backoff // for up to about 30 days, with a maximum interval of about // 24 hours. Cancelling ctx will cancel retries and shut down // any goroutines spawned by ManageAsync. func (cfg *Config) ManageAsync(ctx context.Context, domainNames []string) error { return cfg.manageAll(ctx, domainNames, true) } // ClientCredentials returns a list of TLS client certificate chains for the given identifiers. // The return value can be used in a tls.Config to enable client authentication using managed certificates. // Any certificates that need to be obtained or renewed for these identifiers will be managed accordingly. func (cfg *Config) ClientCredentials(ctx context.Context, identifiers []string) ([]tls.Certificate, error) { err := cfg.manageAll(ctx, identifiers, false) if err != nil { return nil, err } var chains []tls.Certificate for _, id := range identifiers { certRes, err := cfg.loadCertResourceAnyIssuer(id) if err != nil { return chains, err } chain, err := tls.X509KeyPair(certRes.CertificatePEM, certRes.PrivateKeyPEM) if err != nil { return chains, err } chains = append(chains, chain) } return chains, nil } func (cfg *Config) manageAll(ctx context.Context, domainNames []string, async bool) error { if ctx == nil { ctx = context.Background() } for _, domainName := range domainNames { // if on-demand is configured, defer obtain and renew operations if cfg.OnDemand != nil { if !cfg.OnDemand.whitelistContains(domainName) { cfg.OnDemand.hostWhitelist = append(cfg.OnDemand.hostWhitelist, domainName) } continue } // otherwise, begin management immediately err := cfg.manageOne(ctx, domainName, async) if err != nil { return err } } return nil } func (cfg *Config) manageOne(ctx context.Context, domainName string, async bool) error { // first try loading existing certificate from storage cert, err := cfg.CacheManagedCertificate(domainName) if err != nil { if _, ok := err.(ErrNotExist); !ok { return fmt.Errorf("%s: caching certificate: %v", domainName, err) } // if we don't have one in storage, obtain one obtain := func() error { var err error if async { err = cfg.ObtainCertAsync(ctx, domainName) } else { err = cfg.ObtainCertSync(ctx, domainName) } if err != nil { return fmt.Errorf("%s: obtaining certificate: %w", domainName, err) } cert, err = cfg.CacheManagedCertificate(domainName) if err != nil { return fmt.Errorf("%s: caching certificate after obtaining it: %v", domainName, err) } return nil } if async { // Leave the job name empty so as to allow duplicate 'obtain' // jobs; this is because Caddy calls ManageAsync() before the // previous config is stopped (and before its context is // canceled), which means that if an obtain job is still // running for the same domain, Submit() would not queue the // new one because it is still running, even though it is // (probably) about to be canceled (it might not if the new // config fails to finish loading, however). In any case, we // presume it is safe to enqueue a duplicate obtain job because // either the old one (or sometimes the new one) is about to be // canceled. This seems like reasonable logic for any consumer // of this lib. See https://github.com/caddyserver/caddy/issues/3202 jm.Submit(cfg.Logger, "", obtain) return nil } return obtain() } // for an existing certificate, make sure it is renewed renew := func() error { var err error if async { err = cfg.RenewCertAsync(ctx, domainName, false) } else { err = cfg.RenewCertSync(ctx, domainName, false) } if err != nil { return fmt.Errorf("%s: renewing certificate: %w", domainName, err) } // successful renewal, so update in-memory cache err = cfg.reloadManagedCertificate(cert) if err != nil { return fmt.Errorf("%s: reloading renewed certificate into memory: %v", domainName, err) } return nil } if cert.NeedsRenewal(cfg) { if async { jm.Submit(cfg.Logger, "renew_"+domainName, renew) return nil } return renew() } return nil } // Unmanage causes the certificates for domainNames to stop being managed. // If there are certificates for the supplied domain names in the cache, they // are evicted from the cache. func (cfg *Config) Unmanage(domainNames []string) { var deleteQueue []Certificate for _, domainName := range domainNames { certs := cfg.certCache.AllMatchingCertificates(domainName) for _, cert := range certs { if !cert.managed { continue } deleteQueue = append(deleteQueue, cert) } } cfg.certCache.mu.Lock() for _, cert := range deleteQueue { cfg.certCache.removeCertificate(cert) } cfg.certCache.mu.Unlock() } // ObtainCertSync generates a new private key and obtains a certificate for // name using cfg in the foreground; i.e. interactively and without retries. // It stows the renewed certificate and its assets in storage if successful. // It DOES NOT load the certificate into the in-memory cache. This method // is a no-op if storage already has a certificate for name. func (cfg *Config) ObtainCertSync(ctx context.Context, name string) error { return cfg.obtainCert(ctx, name, true) } // ObtainCertAsync is the same as ObtainCertSync(), except it runs in the // background; i.e. non-interactively, and with retries if it fails. func (cfg *Config) ObtainCertAsync(ctx context.Context, name string) error { return cfg.obtainCert(ctx, name, false) } func (cfg *Config) obtainCert(ctx context.Context, name string, interactive bool) error { if len(cfg.Issuers) == 0 { return fmt.Errorf("no issuers configured; impossible to obtain or check for existing certificate in storage") } // if storage has all resources for this certificate, obtain is a no-op if cfg.storageHasCertResourcesAnyIssuer(name) { return nil } // ensure storage is writeable and readable // TODO: this is not necessary every time; should only perform check once every so often for each storage, which may require some global state... err := cfg.checkStorage() if err != nil { return fmt.Errorf("failed storage check: %v - storage is probably misconfigured", err) } log := loggerNamed(cfg.Logger, "obtain") if log != nil { log.Info("acquiring lock", zap.String("identifier", name)) } // ensure idempotency of the obtain operation for this name lockKey := cfg.lockKey(certIssueLockOp, name) err = acquireLock(ctx, cfg.Storage, lockKey) if err != nil { return fmt.Errorf("unable to acquire lock '%s': %v", lockKey, err) } defer func() { if log != nil { log.Info("releasing lock", zap.String("identifier", name)) } if err := releaseLock(cfg.Storage, lockKey); err != nil { if log != nil { log.Error("unable to unlock", zap.String("identifier", name), zap.String("lock_key", lockKey), zap.Error(err)) } } }() if log != nil { log.Info("lock acquired", zap.String("identifier", name)) } f := func(ctx context.Context) error { // check if obtain is still needed -- might have been obtained during lock if cfg.storageHasCertResourcesAnyIssuer(name) { if log != nil { log.Info("certificate already exists in storage", zap.String("identifier", name)) } return nil } // if storage has a private key already, use it; otherwise, // we'll generate our own privKey, privKeyPEM, issuers, err := cfg.reusePrivateKey(name) if err != nil { return err } if privKey == nil { privKey, err = cfg.KeySource.GenerateKey() if err != nil { return err } privKeyPEM, err = encodePrivateKey(privKey) if err != nil { return err } } csr, err := cfg.generateCSR(privKey, []string{name}) if err != nil { return err } // try to obtain from each issuer until we succeed var issuedCert *IssuedCertificate var issuerUsed Issuer for i, issuer := range issuers { if log != nil { log.Debug(fmt.Sprintf("trying issuer %d/%d", i+1, len(cfg.Issuers)), zap.String("issuer", issuer.IssuerKey())) } if prechecker, ok := issuer.(PreChecker); ok { err = prechecker.PreCheck(ctx, []string{name}, interactive) if err != nil { continue } } issuedCert, err = issuer.Issue(ctx, csr) if err == nil { issuerUsed = issuer break } // err is usually wrapped, which is nice for simply printing it, but // with our structured error logs we only need the problem string errToLog := err var problem acme.Problem if errors.As(err, &problem) { errToLog = problem } if log != nil { log.Error("could not get certificate from issuer", zap.String("identifier", name), zap.String("issuer", issuer.IssuerKey()), zap.Error(errToLog)) } } if err != nil { // only the error from the last issuer will be returned, but we logged the others return fmt.Errorf("[%s] Obtain: %w", name, err) } // success - immediately save the certificate resource certRes := CertificateResource{ SANs: namesFromCSR(csr), CertificatePEM: issuedCert.Certificate, PrivateKeyPEM: privKeyPEM, IssuerData: issuedCert.Metadata, } err = cfg.saveCertResource(issuerUsed, certRes) if err != nil { return fmt.Errorf("[%s] Obtain: saving assets: %v", name, err) } cfg.emit("cert_obtained", name) if log != nil { log.Info("certificate obtained successfully", zap.String("identifier", name)) } return nil } if interactive { err = f(ctx) } else { err = doWithRetry(ctx, log, f) } return err } // reusePrivateKey looks for a private key for domain in storage in the configured issuers // paths. For the first private key it finds, it returns that key both decoded and PEM-encoded, // as well as the reordered list of issuers to use instead of cfg.Issuers (because if a key // is found, that issuer should be tried first, so it is moved to the front in a copy of // cfg.Issuers). func (cfg *Config) reusePrivateKey(domain string) (privKey crypto.PrivateKey, privKeyPEM []byte, issuers []Issuer, err error) { // make a copy of cfg.Issuers so that if we have to reorder elements, we don't // inadvertently mutate the configured issuers (see append calls below) issuers = make([]Issuer, len(cfg.Issuers)) copy(issuers, cfg.Issuers) for i, issuer := range issuers { // see if this issuer location in storage has a private key for the domain privateKeyStorageKey := StorageKeys.SitePrivateKey(issuer.IssuerKey(), domain) privKeyPEM, err = cfg.Storage.Load(privateKeyStorageKey) if _, ok := err.(ErrNotExist); ok { err = nil // obviously, it's OK to not have a private key; so don't prevent obtaining a cert continue } if err != nil { return nil, nil, nil, fmt.Errorf("loading existing private key for reuse with issuer %s: %v", issuer.IssuerKey(), err) } // we loaded a private key; try decoding it so we can use it privKey, err = decodePrivateKey(privKeyPEM) if err != nil { return nil, nil, nil, err } // since the private key was found in storage for this issuer, move it // to the front of the list so we prefer this issuer first issuers = append([]Issuer{issuer}, append(issuers[:i], issuers[i+1:]...)...) break } return } // storageHasCertResourcesAnyIssuer returns true if storage has all the // certificate resources in storage from any configured issuer. It checks // all configured issuers in order. func (cfg *Config) storageHasCertResourcesAnyIssuer(name string) bool { for _, iss := range cfg.Issuers { if cfg.storageHasCertResources(iss, name) { return true } } return false } // RenewCertSync renews the certificate for name using cfg in the foreground; // i.e. interactively and without retries. It stows the renewed certificate // and its assets in storage if successful. It DOES NOT update the in-memory // cache with the new certificate. The certificate will not be renewed if it // is not close to expiring unless force is true. // // Renewing a certificate is the same as obtaining a certificate, except that // the existing private key already in storage is reused. func (cfg *Config) RenewCertSync(ctx context.Context, name string, force bool) error { return cfg.renewCert(ctx, name, force, true) } // RenewCertAsync is the same as RenewCertSync(), except it runs in the // background; i.e. non-interactively, and with retries if it fails. func (cfg *Config) RenewCertAsync(ctx context.Context, name string, force bool) error { return cfg.renewCert(ctx, name, force, false) } func (cfg *Config) renewCert(ctx context.Context, name string, force, interactive bool) error { if len(cfg.Issuers) == 0 { return fmt.Errorf("no issuers configured; impossible to renew or check existing certificate in storage") } // ensure storage is writeable and readable // TODO: this is not necessary every time; should only perform check once every so often for each storage, which may require some global state... err := cfg.checkStorage() if err != nil { return fmt.Errorf("failed storage check: %v - storage is probably misconfigured", err) } log := loggerNamed(cfg.Logger, "renew") if log != nil { log.Info("acquiring lock", zap.String("identifier", name)) } // ensure idempotency of the renew operation for this name lockKey := cfg.lockKey(certIssueLockOp, name) err = acquireLock(ctx, cfg.Storage, lockKey) if err != nil { return fmt.Errorf("unable to acquire lock '%s': %v", lockKey, err) } defer func() { if log != nil { log.Info("releasing lock", zap.String("identifier", name)) } if err := releaseLock(cfg.Storage, lockKey); err != nil { if log != nil { log.Error("unable to unlock", zap.String("identifier", name), zap.String("lock_key", lockKey), zap.Error(err)) } } }() if log != nil { log.Info("lock acquired", zap.String("identifier", name)) } f := func(ctx context.Context) error { // prepare for renewal (load PEM cert, key, and meta) certRes, err := cfg.loadCertResourceAnyIssuer(name) if err != nil { return err } // check if renew is still needed - might have been renewed while waiting for lock timeLeft, needsRenew := cfg.managedCertNeedsRenewal(certRes) if !needsRenew { if force { if log != nil { log.Info("certificate does not need to be renewed, but renewal is being forced", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) } } else { if log != nil { log.Info("certificate appears to have been renewed already", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) } return nil } } if log != nil { log.Info("renewing certificate", zap.String("identifier", name), zap.Duration("remaining", timeLeft)) } privateKey, err := decodePrivateKey(certRes.PrivateKeyPEM) if err != nil { return err } csr, err := cfg.generateCSR(privateKey, []string{name}) if err != nil { return err } // try to obtain from each issuer until we succeed var issuedCert *IssuedCertificate var issuerUsed Issuer for _, issuer := range cfg.Issuers { if prechecker, ok := issuer.(PreChecker); ok { err = prechecker.PreCheck(ctx, []string{name}, interactive) if err != nil { continue } } issuedCert, err = issuer.Issue(ctx, csr) if err == nil { issuerUsed = issuer break } // err is usually wrapped, which is nice for simply printing it, but // with our structured error logs we only need the problem string errToLog := err var problem acme.Problem if errors.As(err, &problem) { errToLog = problem } if log != nil { log.Error("could not get certificate from issuer", zap.String("identifier", name), zap.String("issuer", issuer.IssuerKey()), zap.Error(errToLog)) } } if err != nil { // only the error from the last issuer will be returned, but we logged the others return fmt.Errorf("[%s] Renew: %w", name, err) } // success - immediately save the renewed certificate resource newCertRes := CertificateResource{ SANs: namesFromCSR(csr), CertificatePEM: issuedCert.Certificate, PrivateKeyPEM: certRes.PrivateKeyPEM, IssuerData: issuedCert.Metadata, } err = cfg.saveCertResource(issuerUsed, newCertRes) if err != nil { return fmt.Errorf("[%s] Renew: saving assets: %v", name, err) } cfg.emit("cert_renewed", name) if log != nil { log.Info("certificate renewed successfully", zap.String("identifier", name)) } return nil } if interactive { err = f(ctx) } else { err = doWithRetry(ctx, log, f) } return err } func (cfg *Config) generateCSR(privateKey crypto.PrivateKey, sans []string) (*x509.CertificateRequest, error) { csrTemplate := new(x509.CertificateRequest) for _, name := range sans { if ip := net.ParseIP(name); ip != nil { csrTemplate.IPAddresses = append(csrTemplate.IPAddresses, ip) } else if strings.Contains(name, "@") { csrTemplate.EmailAddresses = append(csrTemplate.EmailAddresses, name) } else if u, err := url.Parse(name); err == nil && strings.Contains(name, "/") { csrTemplate.URIs = append(csrTemplate.URIs, u) } else { // convert IDNs to ASCII according to RFC 5280 section 7 normalizedName, err := idna.ToASCII(name) if err != nil { return nil, fmt.Errorf("converting identifier '%s' to ASCII: %v", name, err) } csrTemplate.DNSNames = append(csrTemplate.DNSNames, normalizedName) } } if cfg.MustStaple { csrTemplate.ExtraExtensions = append(csrTemplate.ExtraExtensions, mustStapleExtension) } csrDER, err := x509.CreateCertificateRequest(rand.Reader, csrTemplate, privateKey) if err != nil { return nil, err } return x509.ParseCertificateRequest(csrDER) } // RevokeCert revokes the certificate for domain via ACME protocol. It requires // that cfg.Issuers is properly configured with the same issuer that issued the // certificate being revoked. See RFC 5280 §5.3.1 for reason codes. // // The certificate assets are deleted from storage after successful revocation // to prevent reuse. func (cfg *Config) RevokeCert(ctx context.Context, domain string, reason int, interactive bool) error { for i, issuer := range cfg.Issuers { issuerKey := issuer.IssuerKey() rev, ok := issuer.(Revoker) if !ok { return fmt.Errorf("issuer %d (%s) is not a Revoker", i, issuerKey) } certRes, err := cfg.loadCertResource(issuer, domain) if err != nil { return err } if !cfg.Storage.Exists(StorageKeys.SitePrivateKey(issuerKey, domain)) { return fmt.Errorf("private key not found for %s", certRes.SANs) } err = rev.Revoke(ctx, certRes, reason) if err != nil { return fmt.Errorf("issuer %d (%s): %v", i, issuerKey, err) } cfg.emit("cert_revoked", domain) err = cfg.deleteSiteAssets(issuerKey, domain) if err != nil { return fmt.Errorf("certificate revoked, but unable to fully clean up assets from issuer %s: %v", issuerKey, err) } } return nil } // TLSConfig is an opinionated method that returns a recommended, modern // TLS configuration that can be used to configure TLS listeners. Aside // from safe, modern defaults, this method sets two critical fields on the // TLS config which are required to enable automatic certificate // management: GetCertificate and NextProtos. // // The GetCertificate field is necessary to get certificates from memory // or storage, including both manual and automated certificates. You // should only change this field if you know what you are doing. // // The NextProtos field is pre-populated with a special value to enable // solving the TLS-ALPN ACME challenge. Because this method does not // assume any particular protocols after the TLS handshake is completed, // you will likely need to customize the NextProtos field by prepending // your application's protocols to the slice. For example, to serve // HTTP, you will need to prepend "h2" and "http/1.1" values. Be sure to // leave the acmez.ACMETLS1Protocol value intact, however, or TLS-ALPN // challenges will fail (which may be acceptable if you are not using // ACME, or specifically, the TLS-ALPN challenge). // // Unlike the package TLS() function, this method does not, by itself, // enable certificate management for any domain names. func (cfg *Config) TLSConfig() *tls.Config { return &tls.Config{ // these two fields necessary for TLS-ALPN challenge GetCertificate: cfg.GetCertificate, NextProtos: []string{acmez.ACMETLS1Protocol}, // the rest recommended for modern TLS servers MinVersion: tls.VersionTLS12, CurvePreferences: []tls.CurveID{ tls.X25519, tls.CurveP256, }, CipherSuites: preferredDefaultCipherSuites(), PreferServerCipherSuites: true, } } // getChallengeInfo loads the challenge info from either the internal challenge memory // or the external storage (implying distributed solving). The second return value // indicates whether challenge info was loaded from external storage. If true, the // challenge is being solved in a distributed fashion; if false, from internal memory. // If no matching challenge information can be found, an error is returned. func (cfg *Config) getChallengeInfo(identifier string) (Challenge, bool, error) { // first, check if our process initiated this challenge; if so, just return it chalData, ok := GetACMEChallenge(identifier) if ok { return chalData, false, nil } // otherwise, perhaps another instance in the cluster initiated it; check // the configured storage to retrieve challenge data var chalInfo acme.Challenge var chalInfoBytes []byte var tokenKey string for _, issuer := range cfg.Issuers { ds := distributedSolver{ storage: cfg.Storage, storageKeyIssuerPrefix: storageKeyACMECAPrefix(issuer.IssuerKey()), } tokenKey = ds.challengeTokensKey(identifier) var err error chalInfoBytes, err = cfg.Storage.Load(tokenKey) if err == nil { break } if _, ok := err.(ErrNotExist); ok { continue } return Challenge{}, false, fmt.Errorf("opening distributed challenge token file %s: %v", tokenKey, err) } if len(chalInfoBytes) == 0 { return Challenge{}, false, fmt.Errorf("no information found to solve challenge for identifier: %s", identifier) } err := json.Unmarshal(chalInfoBytes, &chalInfo) if err != nil { return Challenge{}, false, fmt.Errorf("decoding challenge token file %s (corrupted?): %v", tokenKey, err) } return Challenge{Challenge: chalInfo}, true, nil } // checkStorage tests the storage by writing random bytes // to a random key, and then loading those bytes and // comparing the loaded value. If this fails, the provided // cfg.Storage mechanism should not be used. func (cfg *Config) checkStorage() error { key := fmt.Sprintf("rw_test_%d", weakrand.Int()) contents := make([]byte, 1024*10) // size sufficient for one or two ACME resources _, err := weakrand.Read(contents) if err != nil { return err } err = cfg.Storage.Store(key, contents) if err != nil { return err } defer func() { deleteErr := cfg.Storage.Delete(key) if deleteErr != nil { if cfg.Logger != nil { cfg.Logger.Error("deleting test key from storage", zap.String("key", key), zap.Error(err)) } } // if there was no other error, make sure // to return any error returned from Delete if err == nil { err = deleteErr } }() loaded, err := cfg.Storage.Load(key) if err != nil { return err } if !bytes.Equal(contents, loaded) { return fmt.Errorf("load yielded different value than was stored; expected %d bytes, got %d bytes of differing elements", len(contents), len(loaded)) } return nil } // storageHasCertResources returns true if the storage // associated with cfg's certificate cache has all the // resources related to the certificate for domain: the // certificate, the private key, and the metadata. func (cfg *Config) storageHasCertResources(issuer Issuer, domain string) bool { issuerKey := issuer.IssuerKey() certKey := StorageKeys.SiteCert(issuerKey, domain) keyKey := StorageKeys.SitePrivateKey(issuerKey, domain) metaKey := StorageKeys.SiteMeta(issuerKey, domain) return cfg.Storage.Exists(certKey) && cfg.Storage.Exists(keyKey) && cfg.Storage.Exists(metaKey) } // deleteSiteAssets deletes the folder in storage containing the // certificate, private key, and metadata file for domain from the // issuer with the given issuer key. func (cfg *Config) deleteSiteAssets(issuerKey, domain string) error { err := cfg.Storage.Delete(StorageKeys.SiteCert(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting certificate file: %v", err) } err = cfg.Storage.Delete(StorageKeys.SitePrivateKey(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting private key: %v", err) } err = cfg.Storage.Delete(StorageKeys.SiteMeta(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting metadata file: %v", err) } err = cfg.Storage.Delete(StorageKeys.CertsSitePrefix(issuerKey, domain)) if err != nil { return fmt.Errorf("deleting site asset folder: %v", err) } return nil } // lockKey returns a key for a lock that is specific to the operation // named op being performed related to domainName and this config's CA. func (cfg *Config) lockKey(op, domainName string) string { return fmt.Sprintf("%s_%s", op, domainName) } // managedCertNeedsRenewal returns true if certRes is expiring soon or already expired, // or if the process of decoding the cert and checking its expiration returned an error. func (cfg *Config) managedCertNeedsRenewal(certRes CertificateResource) (time.Duration, bool) { certChain, err := parseCertsFromPEMBundle(certRes.CertificatePEM) if err != nil { return 0, true } remaining := time.Until(certChain[0].NotAfter) needsRenew := currentlyInRenewalWindow(certChain[0].NotBefore, certChain[0].NotAfter, cfg.RenewalWindowRatio) return remaining, needsRenew } func (cfg *Config) emit(eventName string, data interface{}) { if cfg.OnEvent == nil { return } cfg.OnEvent(eventName, data) } func loggerNamed(l *zap.Logger, name string) *zap.Logger { if l == nil { return nil } return l.Named(name) } // CertificateSelector is a type which can select a certificate to use given multiple choices. type CertificateSelector interface { SelectCertificate(*tls.ClientHelloInfo, []Certificate) (Certificate, error) } // OCSPConfig configures how OCSP is handled. type OCSPConfig struct { // Disable automatic OCSP stapling; strongly // discouraged unless you have a good reason. // Disabling this puts clients at greater risk // and reduces their privacy. DisableStapling bool // A map of OCSP responder domains to replacement // domains for querying OCSP servers. Used for // overriding the OCSP responder URL that is // embedded in certificates. Mapping to an empty // URL will disable OCSP from that responder. ResponderOverrides map[string]string } // certIssueLockOp is the name of the operation used // when naming a lock to make it mutually exclusive // with other certificate issuance operations for a // certain name. const certIssueLockOp = "issue_cert" // Constants for PKIX MustStaple extension. var ( tlsFeatureExtensionOID = asn1.ObjectIdentifier{1, 3, 6, 1, 5, 5, 7, 1, 24} ocspMustStapleFeature = []byte{0x30, 0x03, 0x02, 0x01, 0x05} mustStapleExtension = pkix.Extension{ Id: tlsFeatureExtensionOID, Value: ocspMustStapleFeature, } ) golang-github-caddyserver-certmagic-0.15.2/config_test.go000066400000000000000000000040071414454377300234430ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "os" "reflect" "testing" "github.com/mholt/acmez/acme" ) func TestSaveCertResource(t *testing.T) { am := &ACMEManager{CA: "https://example.com/acme/directory"} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() domain := "example.com" certContents := "certificate" keyContents := "private key" cert := CertificateResource{ SANs: []string{domain}, PrivateKeyPEM: []byte(keyContents), CertificatePEM: []byte(certContents), IssuerData: &acme.Certificate{ URL: "https://example.com/cert", }, issuerKey: am.IssuerKey(), } err := testConfig.saveCertResource(am, cert) if err != nil { t.Fatalf("Expected no error, got: %v", err) } // the result of our test will be a map, since we have // no choice but to decode it into an interface cert.IssuerData = map[string]interface{}{ "url": "https://example.com/cert", } siteData, err := testConfig.loadCertResource(am, domain) if err != nil { t.Fatalf("Expected no error reading site, got: %v", err) } if !reflect.DeepEqual(cert, siteData) { t.Errorf("Expected '%+v' to match '%+v'", cert, siteData) } } golang-github-caddyserver-certmagic-0.15.2/crypto.go000066400000000000000000000257141414454377300224670ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto" "crypto/ecdsa" "crypto/ed25519" "crypto/elliptic" "crypto/rand" "crypto/rsa" "crypto/sha256" "crypto/tls" "crypto/x509" "encoding/json" "encoding/pem" "fmt" "hash/fnv" "sort" "strings" "github.com/klauspost/cpuid/v2" "go.uber.org/zap" "golang.org/x/net/idna" ) // encodePrivateKey marshals a EC or RSA private key into a PEM-encoded array of bytes. func encodePrivateKey(key crypto.PrivateKey) ([]byte, error) { var pemType string var keyBytes []byte switch key := key.(type) { case *ecdsa.PrivateKey: var err error pemType = "EC" keyBytes, err = x509.MarshalECPrivateKey(key) if err != nil { return nil, err } case *rsa.PrivateKey: pemType = "RSA" keyBytes = x509.MarshalPKCS1PrivateKey(key) case ed25519.PrivateKey: var err error pemType = "ED25519" keyBytes, err = x509.MarshalPKCS8PrivateKey(key) if err != nil { return nil, err } default: return nil, fmt.Errorf("unsupported key type: %T", key) } pemKey := pem.Block{Type: pemType + " PRIVATE KEY", Bytes: keyBytes} return pem.EncodeToMemory(&pemKey), nil } // decodePrivateKey loads a PEM-encoded ECC/RSA private key from an array of bytes. // Borrowed from Go standard library, to handle various private key and PEM block types. // https://github.com/golang/go/blob/693748e9fa385f1e2c3b91ca9acbb6c0ad2d133d/src/crypto/tls/tls.go#L291-L308 // https://github.com/golang/go/blob/693748e9fa385f1e2c3b91ca9acbb6c0ad2d133d/src/crypto/tls/tls.go#L238) func decodePrivateKey(keyPEMBytes []byte) (crypto.Signer, error) { keyBlockDER, _ := pem.Decode(keyPEMBytes) if keyBlockDER == nil { return nil, fmt.Errorf("failed to decode PEM block containing private key") } if keyBlockDER.Type != "PRIVATE KEY" && !strings.HasSuffix(keyBlockDER.Type, " PRIVATE KEY") { return nil, fmt.Errorf("unknown PEM header %q", keyBlockDER.Type) } if key, err := x509.ParsePKCS1PrivateKey(keyBlockDER.Bytes); err == nil { return key, nil } if key, err := x509.ParsePKCS8PrivateKey(keyBlockDER.Bytes); err == nil { switch key := key.(type) { case *rsa.PrivateKey, *ecdsa.PrivateKey, ed25519.PrivateKey: return key.(crypto.Signer), nil default: return nil, fmt.Errorf("found unknown private key type in PKCS#8 wrapping: %T", key) } } if key, err := x509.ParseECPrivateKey(keyBlockDER.Bytes); err == nil { return key, nil } return nil, fmt.Errorf("unknown private key type") } // parseCertsFromPEMBundle parses a certificate bundle from top to bottom and returns // a slice of x509 certificates. This function will error if no certificates are found. func parseCertsFromPEMBundle(bundle []byte) ([]*x509.Certificate, error) { var certificates []*x509.Certificate var certDERBlock *pem.Block for { certDERBlock, bundle = pem.Decode(bundle) if certDERBlock == nil { break } if certDERBlock.Type == "CERTIFICATE" { cert, err := x509.ParseCertificate(certDERBlock.Bytes) if err != nil { return nil, err } certificates = append(certificates, cert) } } if len(certificates) == 0 { return nil, fmt.Errorf("no certificates found in bundle") } return certificates, nil } // fastHash hashes input using a hashing algorithm that // is fast, and returns the hash as a hex-encoded string. // Do not use this for cryptographic purposes. func fastHash(input []byte) string { h := fnv.New32a() h.Write(input) return fmt.Sprintf("%x", h.Sum32()) } // saveCertResource saves the certificate resource to disk. This // includes the certificate file itself, the private key, and the // metadata file. func (cfg *Config) saveCertResource(issuer Issuer, cert CertificateResource) error { metaBytes, err := json.MarshalIndent(cert, "", "\t") if err != nil { return fmt.Errorf("encoding certificate metadata: %v", err) } issuerKey := issuer.IssuerKey() certKey := cert.NamesKey() all := []keyValue{ { key: StorageKeys.SitePrivateKey(issuerKey, certKey), value: cert.PrivateKeyPEM, }, { key: StorageKeys.SiteCert(issuerKey, certKey), value: cert.CertificatePEM, }, { key: StorageKeys.SiteMeta(issuerKey, certKey), value: metaBytes, }, } return storeTx(cfg.Storage, all) } // loadCertResourceAnyIssuer loads and returns the certificate resource from any // of the configured issuers. If multiple are found (e.g. if there are 3 issuers // configured, and all 3 have a resource matching certNamesKey), then the newest // (latest NotBefore date) resource will be chosen. func (cfg *Config) loadCertResourceAnyIssuer(certNamesKey string) (CertificateResource, error) { // we can save some extra decoding steps if there's only one issuer, since // we don't need to compare potentially multiple available resources to // select the best one, when there's only one choice anyway if len(cfg.Issuers) == 1 { return cfg.loadCertResource(cfg.Issuers[0], certNamesKey) } type decodedCertResource struct { CertificateResource issuer Issuer decoded *x509.Certificate } var certResources []decodedCertResource var lastErr error // load and decode all certificate resources found with the // configured issuers so we can sort by newest for _, issuer := range cfg.Issuers { certRes, err := cfg.loadCertResource(issuer, certNamesKey) if err != nil { if _, ok := err.(ErrNotExist); ok { // not a problem, but we need to remember the error // in case we end up not finding any cert resources // since we'll need an error to return in that case lastErr = err continue } return CertificateResource{}, err } certs, err := parseCertsFromPEMBundle(certRes.CertificatePEM) if err != nil { return CertificateResource{}, err } certResources = append(certResources, decodedCertResource{ CertificateResource: certRes, issuer: issuer, decoded: certs[0], }) } if len(certResources) == 0 { if lastErr == nil { lastErr = fmt.Errorf("no certificate resources found") // just in case; e.g. no Issuers configured } return CertificateResource{}, lastErr } // sort by date so the most recently issued comes first sort.Slice(certResources, func(i, j int) bool { return certResources[j].decoded.NotBefore.Before(certResources[i].decoded.NotBefore) }) if cfg.Logger != nil { cfg.Logger.Debug("loading managed certificate", zap.String("domain", certNamesKey), zap.Time("expiration", certResources[0].decoded.NotAfter), zap.String("issuer_key", certResources[0].issuer.IssuerKey()), zap.Any("storage", cfg.Storage), ) } return certResources[0].CertificateResource, nil } // loadCertResource loads a certificate resource from the given issuer's storage location. func (cfg *Config) loadCertResource(issuer Issuer, certNamesKey string) (CertificateResource, error) { certRes := CertificateResource{issuerKey: issuer.IssuerKey()} normalizedName, err := idna.ToASCII(certNamesKey) if err != nil { return CertificateResource{}, fmt.Errorf("converting '%s' to ASCII: %v", certNamesKey, err) } certBytes, err := cfg.Storage.Load(StorageKeys.SiteCert(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } certRes.CertificatePEM = certBytes keyBytes, err := cfg.Storage.Load(StorageKeys.SitePrivateKey(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } certRes.PrivateKeyPEM = keyBytes metaBytes, err := cfg.Storage.Load(StorageKeys.SiteMeta(certRes.issuerKey, normalizedName)) if err != nil { return CertificateResource{}, err } err = json.Unmarshal(metaBytes, &certRes) if err != nil { return CertificateResource{}, fmt.Errorf("decoding certificate metadata: %v", err) } return certRes, nil } // hashCertificateChain computes the unique hash of certChain, // which is the chain of DER-encoded bytes. It returns the // hex encoding of the hash. func hashCertificateChain(certChain [][]byte) string { h := sha256.New() for _, certInChain := range certChain { h.Write(certInChain) } return fmt.Sprintf("%x", h.Sum(nil)) } func namesFromCSR(csr *x509.CertificateRequest) []string { var nameSet []string nameSet = append(nameSet, csr.DNSNames...) nameSet = append(nameSet, csr.EmailAddresses...) for _, v := range csr.IPAddresses { nameSet = append(nameSet, v.String()) } for _, v := range csr.URIs { nameSet = append(nameSet, v.String()) } return nameSet } // preferredDefaultCipherSuites returns an appropriate // cipher suite to use depending on hardware support // for AES-NI. // // See https://github.com/mholt/caddy/issues/1674 func preferredDefaultCipherSuites() []uint16 { if cpuid.CPU.Supports(cpuid.AESNI) { return defaultCiphersPreferAES } return defaultCiphersPreferChaCha } var ( defaultCiphersPreferAES = []uint16{ tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, } defaultCiphersPreferChaCha = []uint16{ tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, } ) // StandardKeyGenerator is the standard, in-memory key source // that uses crypto/rand. type StandardKeyGenerator struct { // The type of keys to generate. KeyType KeyType } // GenerateKey generates a new private key according to kg.KeyType. func (kg StandardKeyGenerator) GenerateKey() (crypto.PrivateKey, error) { switch kg.KeyType { case ED25519: _, priv, err := ed25519.GenerateKey(rand.Reader) return priv, err case "", P256: return ecdsa.GenerateKey(elliptic.P256(), rand.Reader) case P384: return ecdsa.GenerateKey(elliptic.P384(), rand.Reader) case RSA2048: return rsa.GenerateKey(rand.Reader, 2048) case RSA4096: return rsa.GenerateKey(rand.Reader, 4096) case RSA8192: return rsa.GenerateKey(rand.Reader, 8192) } return nil, fmt.Errorf("unrecognized or unsupported key type: %s", kg.KeyType) } // DefaultKeyGenerator is the default key source. var DefaultKeyGenerator = StandardKeyGenerator{KeyType: P256} // KeyType enumerates the known/supported key types. type KeyType string // Constants for all key types we support. const ( ED25519 = KeyType("ed25519") P256 = KeyType("p256") P384 = KeyType("p384") RSA2048 = KeyType("rsa2048") RSA4096 = KeyType("rsa4096") RSA8192 = KeyType("rsa8192") ) golang-github-caddyserver-certmagic-0.15.2/crypto_test.go000066400000000000000000000050661414454377300235240ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "crypto" "crypto/ecdsa" "crypto/ed25519" "crypto/elliptic" "crypto/rand" "crypto/rsa" "crypto/x509" "testing" ) func TestEncodeDecodeRSAPrivateKey(t *testing.T) { privateKey, err := rsa.GenerateKey(rand.Reader, 128) // make tests faster; small key size OK for testing if err != nil { t.Fatal(err) } // test save savedBytes, err := encodePrivateKey(privateKey) if err != nil { t.Fatal("error saving private key:", err) } // test load loadedKey, err := decodePrivateKey(savedBytes) if err != nil { t.Error("error loading private key:", err) } // test load (should fail) _, err = decodePrivateKey(savedBytes[2:]) if err == nil { t.Error("loading private key should have failed") } // verify loaded key is correct if !privateKeysSame(privateKey, loadedKey) { t.Error("Expected key bytes to be the same, but they weren't") } } func TestSaveAndLoadECCPrivateKey(t *testing.T) { privateKey, err := ecdsa.GenerateKey(elliptic.P384(), rand.Reader) if err != nil { t.Fatal(err) } // test save savedBytes, err := encodePrivateKey(privateKey) if err != nil { t.Fatal("error saving private key:", err) } // test load loadedKey, err := decodePrivateKey(savedBytes) if err != nil { t.Error("error loading private key:", err) } // verify loaded key is correct if !privateKeysSame(privateKey, loadedKey) { t.Error("Expected key bytes to be the same, but they weren't") } } // privateKeysSame compares the bytes of a and b and returns true if they are the same. func privateKeysSame(a, b crypto.PrivateKey) bool { return bytes.Equal(privateKeyBytes(a), privateKeyBytes(b)) } // privateKeyBytes returns the bytes of DER-encoded key. func privateKeyBytes(key crypto.PrivateKey) []byte { var keyBytes []byte switch key := key.(type) { case *rsa.PrivateKey: keyBytes = x509.MarshalPKCS1PrivateKey(key) case *ecdsa.PrivateKey: keyBytes, _ = x509.MarshalECPrivateKey(key) case ed25519.PrivateKey: return key } return keyBytes } golang-github-caddyserver-certmagic-0.15.2/dnsutil.go000066400000000000000000000211301414454377300226150ustar00rootroot00000000000000package certmagic import ( "errors" "fmt" "net" "strings" "sync" "time" "github.com/miekg/dns" ) // Code in this file adapted from go-acme/lego, July 2020: // https://github.com/go-acme/lego // by Ludovic Fernandez and Dominik Menke // // It has been modified. // findZoneByFQDN determines the zone apex for the given fqdn by recursing // up the domain labels until the nameserver returns a SOA record in the // answer section. func findZoneByFQDN(fqdn string, nameservers []string) (string, error) { if !strings.HasSuffix(fqdn, ".") { fqdn += "." } soa, err := lookupSoaByFqdn(fqdn, nameservers) if err != nil { return "", err } return soa.zone, nil } func lookupSoaByFqdn(fqdn string, nameservers []string) (*soaCacheEntry, error) { if !strings.HasSuffix(fqdn, ".") { fqdn += "." } fqdnSOACacheMu.Lock() defer fqdnSOACacheMu.Unlock() // prefer cached version if fresh if ent := fqdnSOACache[fqdn]; ent != nil && !ent.isExpired() { return ent, nil } ent, err := fetchSoaByFqdn(fqdn, nameservers) if err != nil { return nil, err } // save result to cache, but don't allow // the cache to grow out of control if len(fqdnSOACache) >= 1000 { for key := range fqdnSOACache { delete(fqdnSOACache, key) break } } fqdnSOACache[fqdn] = ent return ent, nil } func fetchSoaByFqdn(fqdn string, nameservers []string) (*soaCacheEntry, error) { var err error var in *dns.Msg labelIndexes := dns.Split(fqdn) for _, index := range labelIndexes { domain := fqdn[index:] in, err = dnsQuery(domain, dns.TypeSOA, nameservers, true) if err != nil { continue } if in == nil { continue } switch in.Rcode { case dns.RcodeSuccess: // Check if we got a SOA RR in the answer section if len(in.Answer) == 0 { continue } // CNAME records cannot/should not exist at the root of a zone. // So we skip a domain when a CNAME is found. if dnsMsgContainsCNAME(in) { continue } for _, ans := range in.Answer { if soa, ok := ans.(*dns.SOA); ok { return newSoaCacheEntry(soa), nil } } case dns.RcodeNameError: // NXDOMAIN default: // Any response code other than NOERROR and NXDOMAIN is treated as error return nil, fmt.Errorf("unexpected response code '%s' for %s", dns.RcodeToString[in.Rcode], domain) } } return nil, fmt.Errorf("could not find the start of authority for %s%s", fqdn, formatDNSError(in, err)) } // dnsMsgContainsCNAME checks for a CNAME answer in msg func dnsMsgContainsCNAME(msg *dns.Msg) bool { for _, ans := range msg.Answer { if _, ok := ans.(*dns.CNAME); ok { return true } } return false } func dnsQuery(fqdn string, rtype uint16, nameservers []string, recursive bool) (*dns.Msg, error) { m := createDNSMsg(fqdn, rtype, recursive) var in *dns.Msg var err error for _, ns := range nameservers { in, err = sendDNSQuery(m, ns) if err == nil && len(in.Answer) > 0 { break } } return in, err } func createDNSMsg(fqdn string, rtype uint16, recursive bool) *dns.Msg { m := new(dns.Msg) m.SetQuestion(fqdn, rtype) m.SetEdns0(4096, false) if !recursive { m.RecursionDesired = false } return m } func sendDNSQuery(m *dns.Msg, ns string) (*dns.Msg, error) { udp := &dns.Client{Net: "udp", Timeout: dnsTimeout} in, _, err := udp.Exchange(m, ns) // two kinds of errors we can handle by retrying with TCP: // truncation and timeout; see https://github.com/caddyserver/caddy/issues/3639 truncated := in != nil && in.Truncated timeoutErr := err != nil && strings.Contains(err.Error(), "timeout") if truncated || timeoutErr { tcp := &dns.Client{Net: "tcp", Timeout: dnsTimeout} in, _, err = tcp.Exchange(m, ns) } return in, err } func formatDNSError(msg *dns.Msg, err error) string { var parts []string if msg != nil { parts = append(parts, dns.RcodeToString[msg.Rcode]) } if err != nil { parts = append(parts, err.Error()) } if len(parts) > 0 { return ": " + strings.Join(parts, " ") } return "" } // soaCacheEntry holds a cached SOA record (only selected fields) type soaCacheEntry struct { zone string // zone apex (a domain name) primaryNs string // primary nameserver for the zone apex expires time.Time // time when this cache entry should be evicted } func newSoaCacheEntry(soa *dns.SOA) *soaCacheEntry { return &soaCacheEntry{ zone: soa.Hdr.Name, primaryNs: soa.Ns, expires: time.Now().Add(time.Duration(soa.Refresh) * time.Second), } } // isExpired checks whether a cache entry should be considered expired. func (cache *soaCacheEntry) isExpired() bool { return time.Now().After(cache.expires) } // systemOrDefaultNameservers attempts to get system nameservers from the // resolv.conf file given by path before falling back to hard-coded defaults. func systemOrDefaultNameservers(path string, defaults []string) []string { config, err := dns.ClientConfigFromFile(path) if err != nil || len(config.Servers) == 0 { return defaults } return config.Servers } // populateNameserverPorts ensures that all nameservers have a port number. func populateNameserverPorts(servers []string) { for i := range servers { _, port, _ := net.SplitHostPort(servers[i]) if port == "" { servers[i] = net.JoinHostPort(servers[i], "53") } } } // checkDNSPropagation checks if the expected TXT record has been propagated to all authoritative nameservers. func checkDNSPropagation(fqdn, value string, resolvers []string) (bool, error) { if !strings.HasSuffix(fqdn, ".") { fqdn += "." } // Initial attempt to resolve at the recursive NS r, err := dnsQuery(fqdn, dns.TypeTXT, resolvers, true) if err != nil { return false, err } // TODO: make this configurable, maybe // if !p.requireCompletePropagation { // return true, nil // } if r.Rcode == dns.RcodeSuccess { fqdn = updateDomainWithCName(r, fqdn) } authoritativeNss, err := lookupNameservers(fqdn, resolvers) if err != nil { return false, err } return checkAuthoritativeNss(fqdn, value, authoritativeNss) } // checkAuthoritativeNss queries each of the given nameservers for the expected TXT record. func checkAuthoritativeNss(fqdn, value string, nameservers []string) (bool, error) { for _, ns := range nameservers { r, err := dnsQuery(fqdn, dns.TypeTXT, []string{net.JoinHostPort(ns, "53")}, false) if err != nil { return false, err } if r.Rcode != dns.RcodeSuccess { if r.Rcode == dns.RcodeNameError { // if Present() succeeded, then it must show up eventually, or else // something is really broken in the DNS provider or their API; // no need for error here, simply have the caller try again return false, nil } return false, fmt.Errorf("NS %s returned %s for %s", ns, dns.RcodeToString[r.Rcode], fqdn) } var found bool for _, rr := range r.Answer { if txt, ok := rr.(*dns.TXT); ok { record := strings.Join(txt.Txt, "") if record == value { found = true break } } } if !found { return false, nil } } return true, nil } // lookupNameservers returns the authoritative nameservers for the given fqdn. func lookupNameservers(fqdn string, resolvers []string) ([]string, error) { var authoritativeNss []string zone, err := findZoneByFQDN(fqdn, resolvers) if err != nil { return nil, fmt.Errorf("could not determine the zone: %w", err) } r, err := dnsQuery(zone, dns.TypeNS, resolvers, true) if err != nil { return nil, err } for _, rr := range r.Answer { if ns, ok := rr.(*dns.NS); ok { authoritativeNss = append(authoritativeNss, strings.ToLower(ns.Ns)) } } if len(authoritativeNss) > 0 { return authoritativeNss, nil } return nil, errors.New("could not determine authoritative nameservers") } // Update FQDN with CNAME if any func updateDomainWithCName(r *dns.Msg, fqdn string) string { for _, rr := range r.Answer { if cn, ok := rr.(*dns.CNAME); ok { if cn.Hdr.Name == fqdn { return cn.Target } } } return fqdn } // recursiveNameservers are used to pre-check DNS propagation. It // picks user-configured nameservers (custom) OR the defaults // obtained from resolv.conf and defaultNameservers if none is // configured and ensures that all server addresses have a port value. func recursiveNameservers(custom []string) []string { var servers []string if len(custom) == 0 { servers = systemOrDefaultNameservers(defaultResolvConf, defaultNameservers) } else { servers = make([]string, len(custom)) copy(servers, custom) } populateNameserverPorts(servers) return servers } var defaultNameservers = []string{ "8.8.8.8:53", "8.8.4.4:53", "1.1.1.1:53", "1.0.0.1:53", } var dnsTimeout = 10 * time.Second var ( fqdnSOACache = map[string]*soaCacheEntry{} fqdnSOACacheMu sync.Mutex ) const defaultResolvConf = "/etc/resolv.conf" golang-github-caddyserver-certmagic-0.15.2/dnsutil_test.go000066400000000000000000000151411414454377300236610ustar00rootroot00000000000000package certmagic // Code in this file adapted from go-acme/lego, July 2020: // https://github.com/go-acme/lego // by Ludovic Fernandez and Dominik Menke // // It has been modified. import ( "net" "reflect" "sort" "strings" "testing" ) func TestLookupNameserversOK(t *testing.T) { testCases := []struct { fqdn string nss []string }{ { fqdn: "books.google.com.ng.", nss: []string{"ns1.google.com.", "ns2.google.com.", "ns3.google.com.", "ns4.google.com."}, }, { fqdn: "www.google.com.", nss: []string{"ns1.google.com.", "ns2.google.com.", "ns3.google.com.", "ns4.google.com."}, }, { fqdn: "physics.georgetown.edu.", nss: []string{"ns4.georgetown.edu.", "ns5.georgetown.edu.", "ns6.georgetown.edu."}, }, } for i, test := range testCases { test := test t.Run(test.fqdn, func(t *testing.T) { t.Parallel() nss, err := lookupNameservers(test.fqdn, recursiveNameservers(nil)) if err != nil { t.Errorf("Expected no error, got: %v", err) } sort.Strings(nss) sort.Strings(test.nss) if !reflect.DeepEqual(test.nss, nss) { t.Errorf("Test %d: expected %+v but got %+v", i, test.nss, nss) } }) } } func TestLookupNameserversErr(t *testing.T) { testCases := []struct { desc string fqdn string error string }{ { desc: "invalid tld", fqdn: "_null.n0n0.", error: "could not determine the zone", }, } for i, test := range testCases { test := test t.Run(test.desc, func(t *testing.T) { t.Parallel() _, err := lookupNameservers(test.fqdn, nil) if err == nil { t.Errorf("expected error, got none") } if !strings.Contains(err.Error(), test.error) { t.Errorf("Test %d: Expected error to contain '%s' but got '%s'", i, test.error, err.Error()) } }) } } var findXByFqdnTestCases = []struct { desc string fqdn string zone string primaryNs string nameservers []string expectedError string }{ { desc: "domain is a CNAME", fqdn: "scholar.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: recursiveNameservers(nil), }, { desc: "domain is a non-existent subdomain", fqdn: "foo.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: recursiveNameservers(nil), }, { desc: "domain is a eTLD", fqdn: "example.com.ac.", zone: "ac.", primaryNs: "a0.nic.ac.", nameservers: recursiveNameservers(nil), }, { desc: "domain is a cross-zone CNAME", fqdn: "cross-zone-example.assets.sh.", zone: "assets.sh.", primaryNs: "gina.ns.cloudflare.com.", nameservers: recursiveNameservers(nil), }, { desc: "NXDOMAIN", fqdn: "test.loho.jkl.", zone: "loho.jkl.", nameservers: []string{"1.1.1.1:53"}, expectedError: "could not find the start of authority for test.loho.jkl.: NXDOMAIN", }, { desc: "several non existent nameservers", fqdn: "scholar.google.com.", zone: "google.com.", primaryNs: "ns1.google.com.", nameservers: []string{":7053", ":8053", "1.1.1.1:53"}, }, { desc: "only non existent nameservers", fqdn: "scholar.google.com.", zone: "google.com.", nameservers: []string{":7053", ":8053", ":9053"}, expectedError: "could not find the start of authority for scholar.google.com.:", }, { desc: "no nameservers", fqdn: "test.ldez.com.", zone: "ldez.com.", nameservers: []string{}, expectedError: "could not find the start of authority for test.ldez.com.", }, } func TestFindZoneByFqdn(t *testing.T) { for i, test := range findXByFqdnTestCases { t.Run(test.desc, func(t *testing.T) { clearFqdnCache() zone, err := findZoneByFQDN(test.fqdn, test.nameservers) if test.expectedError != "" { if err == nil { t.Errorf("test %d: expected error, got none", i) return } if !strings.Contains(err.Error(), test.expectedError) { t.Errorf("test %d: expected error to contain '%s' but got '%s'", i, test.expectedError, err.Error()) } } else { if err != nil { t.Errorf("test %d: expected no error, but got: %v", i, err) } if zone != test.zone { t.Errorf("test %d: expected zone '%s' but got '%s'", i, zone, test.zone) } } }) } } func TestResolveConfServers(t *testing.T) { var testCases = []struct { fixture string expected []string defaults []string }{ { fixture: "testdata/resolv.conf.1", defaults: []string{"127.0.0.1:53"}, expected: []string{"10.200.3.249", "10.200.3.250:5353", "2001:4860:4860::8844", "[10.0.0.1]:5353"}, }, { fixture: "testdata/resolv.conf.nonexistant", defaults: []string{"127.0.0.1:53"}, expected: []string{"127.0.0.1:53"}, }, } for i, test := range testCases { t.Run(test.fixture, func(t *testing.T) { result := systemOrDefaultNameservers(test.fixture, test.defaults) sort.Strings(result) sort.Strings(test.expected) if !reflect.DeepEqual(test.expected, result) { t.Errorf("Test %d: Expected %v but got %v", i, test.expected, result) } }) } } func TestRecursiveNameserversAddsPort(t *testing.T) { type want struct { port string } custom := []string{"127.0.0.1", "ns1.google.com:43"} expectations := []want{{port: "53"}, {port: "43"}} results := recursiveNameservers(custom) if !reflect.DeepEqual(custom, []string{"127.0.0.1", "ns1.google.com:43"}) { t.Errorf("Expected custom nameservers to be unmodified. got %v", custom) } if len(results) != len(expectations) { t.Errorf("%v wrong results length. got %d, want %d", results, len(results), len(expectations)) } var hasCustom bool for i, res := range results { hasCustom = hasCustom || strings.HasPrefix(res, custom[0]) if _, port, err := net.SplitHostPort(res); err != nil { t.Errorf("%v Error splitting result %d into host and port: %v", results, i, err) } else { if port != expectations[i].port { t.Errorf("%v Expected result %d to have port %s but got %s", results, i, expectations[i].port, port) } } } if !hasCustom { t.Errorf("%v Expected custom resolvers to be included, but they weren't: %v", results, custom) } } func TestRecursiveNameserversDefaults(t *testing.T) { results := recursiveNameservers(nil) if len(results) < 1 { t.Errorf("%v Expected at least 1 records as default when nil custom", results) } results = recursiveNameservers([]string{}) if len(results) < 1 { t.Errorf("%v Expected at least 1 records as default when empty custom", results) } } func clearFqdnCache() { fqdnSOACacheMu.Lock() fqdnSOACache = make(map[string]*soaCacheEntry) fqdnSOACacheMu.Unlock() } golang-github-caddyserver-certmagic-0.15.2/doc_test.go000066400000000000000000000020701414454377300227410ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "fmt" "log" "net/http" ) // This is the simplest way for HTTP servers to use this package. // Call HTTPS() with your domain names and your handler (or nil // for the http.DefaultMux), and CertMagic will do the rest. func ExampleHTTPS() { http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) { fmt.Fprintf(w, "Hello, HTTPS visitor!") }) err := HTTPS([]string{"example.com", "www.example.com"}, nil) if err != nil { log.Fatal(err) } } golang-github-caddyserver-certmagic-0.15.2/filestorage.go000066400000000000000000000242161414454377300234470ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "encoding/json" "fmt" "io" "io/ioutil" "log" "os" "path" "path/filepath" "runtime" "time" ) // FileStorage facilitates forming file paths derived from a root // directory. It is used to get file paths in a consistent, // cross-platform way or persisting ACME assets on the file system. type FileStorage struct { Path string } // Exists returns true if key exists in fs. func (fs *FileStorage) Exists(key string) bool { _, err := os.Stat(fs.Filename(key)) return !os.IsNotExist(err) } // Store saves value at key. func (fs *FileStorage) Store(key string, value []byte) error { filename := fs.Filename(key) err := os.MkdirAll(filepath.Dir(filename), 0700) if err != nil { return err } return ioutil.WriteFile(filename, value, 0600) } // Load retrieves the value at key. func (fs *FileStorage) Load(key string) ([]byte, error) { contents, err := ioutil.ReadFile(fs.Filename(key)) if os.IsNotExist(err) { return nil, ErrNotExist(err) } return contents, nil } // Delete deletes the value at key. func (fs *FileStorage) Delete(key string) error { err := os.Remove(fs.Filename(key)) if os.IsNotExist(err) { return ErrNotExist(err) } return err } // List returns all keys that match prefix. func (fs *FileStorage) List(prefix string, recursive bool) ([]string, error) { var keys []string walkPrefix := fs.Filename(prefix) err := filepath.Walk(walkPrefix, func(fpath string, info os.FileInfo, err error) error { if err != nil { return err } if info == nil { return fmt.Errorf("%s: file info is nil", fpath) } if fpath == walkPrefix { return nil } suffix, err := filepath.Rel(walkPrefix, fpath) if err != nil { return fmt.Errorf("%s: could not make path relative: %v", fpath, err) } keys = append(keys, path.Join(prefix, suffix)) if !recursive && info.IsDir() { return filepath.SkipDir } return nil }) return keys, err } // Stat returns information about key. func (fs *FileStorage) Stat(key string) (KeyInfo, error) { fi, err := os.Stat(fs.Filename(key)) if os.IsNotExist(err) { return KeyInfo{}, ErrNotExist(err) } if err != nil { return KeyInfo{}, err } return KeyInfo{ Key: key, Modified: fi.ModTime(), Size: fi.Size(), IsTerminal: !fi.IsDir(), }, nil } // Filename returns the key as a path on the file // system prefixed by fs.Path. func (fs *FileStorage) Filename(key string) string { return filepath.Join(fs.Path, filepath.FromSlash(key)) } // Lock obtains a lock named by the given key. It blocks // until the lock can be obtained or an error is returned. func (fs *FileStorage) Lock(ctx context.Context, key string) error { filename := fs.lockFilename(key) for { err := createLockfile(filename) if err == nil { // got the lock, yay return nil } if !os.IsExist(err) { // unexpected error return fmt.Errorf("creating lock file: %v", err) } // lock file already exists var meta lockMeta f, err := os.Open(filename) if err == nil { err2 := json.NewDecoder(f).Decode(&meta) f.Close() if err2 != nil { return fmt.Errorf("decoding lockfile contents: %w", err2) } } switch { case os.IsNotExist(err): // must have just been removed; try again to create it continue case err != nil: // unexpected error return fmt.Errorf("accessing lock file: %v", err) case fileLockIsStale(meta): // lock file is stale - delete it and try again to create one log.Printf("[INFO][%s] Lock for '%s' is stale (created: %s, last update: %s); removing then retrying: %s", fs, key, meta.Created, meta.Updated, filename) removeLockfile(filename) continue default: // lockfile exists and is not stale; // just wait a moment and try again, // or return if context cancelled select { case <-time.After(fileLockPollInterval): case <-ctx.Done(): return ctx.Err() } } } } // Unlock releases the lock for name. func (fs *FileStorage) Unlock(key string) error { return removeLockfile(fs.lockFilename(key)) } func (fs *FileStorage) String() string { return "FileStorage:" + fs.Path } func (fs *FileStorage) lockFilename(key string) string { return filepath.Join(fs.lockDir(), StorageKeys.Safe(key)+".lock") } func (fs *FileStorage) lockDir() string { return filepath.Join(fs.Path, "locks") } func fileLockIsStale(meta lockMeta) bool { ref := meta.Updated if ref.IsZero() { ref = meta.Created } // since updates are exactly every lockFreshnessInterval, // add a grace period for the actual file read+write to // take place return time.Since(ref) > lockFreshnessInterval*2 } // createLockfile atomically creates the lockfile // identified by filename. A successfully created // lockfile should be removed with removeLockfile. func createLockfile(filename string) error { err := atomicallyCreateFile(filename, true) if err != nil { return err } go keepLockfileFresh(filename) // if the app crashes in removeLockfile(), there is a // small chance the .unlock file is left behind; it's // safe to simply remove it as it's a guard against // double removal of the .lock file. _ = os.Remove(filename + ".unlock") return nil } // removeLockfile atomically removes filename, // which must be a lockfile created by createLockfile. // See discussion in PR #7 for more background: // https://github.com/caddyserver/certmagic/pull/7 func removeLockfile(filename string) error { unlockFilename := filename + ".unlock" if err := atomicallyCreateFile(unlockFilename, false); err != nil { if os.IsExist(err) { // another process is handling the unlocking return nil } return err } defer os.Remove(unlockFilename) return os.Remove(filename) } // keepLockfileFresh continuously updates the lock file // at filename with the current timestamp. It stops // when the file disappears (happy path = lock released), // or when there is an error at any point. Since it polls // every lockFreshnessInterval, this function might // not terminate until up to lockFreshnessInterval after // the lock is released. func keepLockfileFresh(filename string) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: active locking: %v\n%s", err, buf) } }() for { time.Sleep(lockFreshnessInterval) done, err := updateLockfileFreshness(filename) if err != nil { log.Printf("[ERROR] Keeping lock file fresh: %v - terminating lock maintenance (lockfile: %s)", err, filename) return } if done { return } } } // updateLockfileFreshness updates the lock file at filename // with the current timestamp. It returns true if the parent // loop can terminate (i.e. no more need to update the lock). func updateLockfileFreshness(filename string) (bool, error) { f, err := os.OpenFile(filename, os.O_RDWR, 0644) if os.IsNotExist(err) { return true, nil // lock released } if err != nil { return true, err } defer f.Close() // read contents metaBytes, err := ioutil.ReadAll(io.LimitReader(f, 2048)) if err != nil { return true, err } var meta lockMeta if err := json.Unmarshal(metaBytes, &meta); err != nil { return true, err } // truncate file and reset I/O offset to beginning if err := f.Truncate(0); err != nil { return true, err } if _, err := f.Seek(0, 0); err != nil { return true, err } // write updated timestamp meta.Updated = time.Now() if err = json.NewEncoder(f).Encode(meta); err != nil { return false, err } // sync to device; we suspect that sometimes file systems // (particularly AWS EFS) don't do this on their own, // leaving the file empty when we close it; see // https://github.com/caddyserver/caddy/issues/3954 return false, f.Sync() } // atomicallyCreateFile atomically creates the file // identified by filename if it doesn't already exist. func atomicallyCreateFile(filename string, writeLockInfo bool) error { // no need to check this error, we only really care about the file creation error _ = os.MkdirAll(filepath.Dir(filename), 0700) f, err := os.OpenFile(filename, os.O_CREATE|os.O_WRONLY|os.O_EXCL, 0644) if err != nil { return err } defer f.Close() if writeLockInfo { now := time.Now() meta := lockMeta{ Created: now, Updated: now, } if err := json.NewEncoder(f).Encode(meta); err != nil { return err } // see https://github.com/caddyserver/caddy/issues/3954 if err := f.Sync(); err != nil { return err } } return nil } // homeDir returns the best guess of the current user's home // directory from environment variables. If unknown, "." (the // current directory) is returned instead. func homeDir() string { home := os.Getenv("HOME") if home == "" && runtime.GOOS == "windows" { drive := os.Getenv("HOMEDRIVE") path := os.Getenv("HOMEPATH") home = drive + path if drive == "" || path == "" { home = os.Getenv("USERPROFILE") } } if home == "" { home = "." } return home } func dataDir() string { baseDir := filepath.Join(homeDir(), ".local", "share") if xdgData := os.Getenv("XDG_DATA_HOME"); xdgData != "" { baseDir = xdgData } return filepath.Join(baseDir, "certmagic") } // lockMeta is written into a lock file. type lockMeta struct { Created time.Time `json:"created,omitempty"` Updated time.Time `json:"updated,omitempty"` } // lockFreshnessInterval is how often to update // a lock's timestamp. Locks with a timestamp // more than this duration in the past (plus a // grace period for latency) can be considered // stale. const lockFreshnessInterval = 5 * time.Second // fileLockPollInterval is how frequently // to check the existence of a lock file const fileLockPollInterval = 1 * time.Second // Interface guard var _ Storage = (*FileStorage)(nil) golang-github-caddyserver-certmagic-0.15.2/go.mod000066400000000000000000000005101414454377300217110ustar00rootroot00000000000000module github.com/caddyserver/certmagic go 1.14 require ( github.com/klauspost/cpuid/v2 v2.0.9 github.com/libdns/libdns v0.2.1 github.com/mholt/acmez v1.0.1 github.com/miekg/dns v1.1.43 go.uber.org/zap v1.17.0 golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a golang.org/x/net v0.0.0-20210525063256-abc453219eb5 ) golang-github-caddyserver-certmagic-0.15.2/go.sum000066400000000000000000000174071414454377300217530ustar00rootroot00000000000000github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4= github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/libdns/libdns v0.2.1 h1:Wu59T7wSHRgtA0cfxC+n1c/e+O3upJGWytknkmFEDis= github.com/libdns/libdns v0.2.1/go.mod h1:yQCXzk1lEZmmCPa857bnk4TsOiqYasqpyOEeSObbb40= github.com/mholt/acmez v1.0.1 h1:J7uquHOKEmo71UDnVApy1sSLA0oF/r+NtVrNzMKKA9I= github.com/mholt/acmez v1.0.1/go.mod h1:8qnn8QA/Ewx8E3ZSsmscqsIjhhpxuy9vqdgbX2ceceM= github.com/miekg/dns v1.1.43 h1:JKfpVSCB84vrAmHzyrsxB5NAr5kLoMXZArPSw7Qlgyg= github.com/miekg/dns v1.1.43/go.mod h1:+evo5L0630/F6ca/Z9+GAqzhjGyn8/c+TBaOyfEl0V4= github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ= go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw= go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU= go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= go.uber.org/multierr v1.6.0/go.mod h1:cdWPpRnG4AhwMwsgIHip0KRBQjJy5kYEpYjJxpXp9iU= go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA= go.uber.org/zap v1.15.0/go.mod h1:Mb2vm2krFEG5DV0W9qcHBYFtp/Wku1cvYaqPsS/WYfc= go.uber.org/zap v1.17.0 h1:MTjgFu6ZLKvY6Pvaqk97GlxNBuMpV4Hy/3P6tRGlI2U= go.uber.org/zap v1.17.0/go.mod h1:MXVU+bhUf/A7Xi2HNOnopQOrmycQ5Ih87HtOu4q5SSo= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a h1:kr2P4QFmQr29mSLA43kwrOcgcReGTfbE9N577tCTuBc= golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8= golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210525063256-abc453219eb5 h1:wjuX4b5yYQnEQHzd+CBcrcC6OVR2J1CN6mUy0oSxIPo= golang.org/x/net v0.0.0-20210525063256-abc453219eb5/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210303074136-134d130e1a04/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da h1:b3NXsE2LusjYGGjL5bxEVZZORm/YEFFrWFjR8eFrw/c= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= golang-github-caddyserver-certmagic-0.15.2/handshake.go000066400000000000000000000626421414454377300230760ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "fmt" "net" "strings" "sync" "time" "github.com/mholt/acmez" "go.uber.org/zap" ) // GetCertificate gets a certificate to satisfy clientHello. In getting // the certificate, it abides the rules and settings defined in the // Config that matches clientHello.ServerName. It first checks the in- // memory cache, then, if the config enables "OnDemand", it accesses // disk, then accesses the network if it must obtain a new certificate // via ACME. // // This method is safe for use as a tls.Config.GetCertificate callback. func (cfg *Config) GetCertificate(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) { cfg.emit("tls_handshake_started", clientHello) // special case: serve up the certificate for a TLS-ALPN ACME challenge // (https://tools.ietf.org/html/draft-ietf-acme-tls-alpn-05) for _, proto := range clientHello.SupportedProtos { if proto == acmez.ACMETLS1Protocol { challengeCert, distributed, err := cfg.getTLSALPNChallengeCert(clientHello) if err != nil { if cfg.Logger != nil { cfg.Logger.Error("tls-alpn challenge", zap.String("server_name", clientHello.ServerName), zap.Error(err)) } return nil, err } if cfg.Logger != nil { cfg.Logger.Info("served key authentication certificate", zap.String("server_name", clientHello.ServerName), zap.String("challenge", "tls-alpn-01"), zap.String("remote", clientHello.Conn.RemoteAddr().String()), zap.Bool("distributed", distributed)) } return challengeCert, nil } } // get the certificate and serve it up cert, err := cfg.getCertDuringHandshake(clientHello, true, true) if err == nil { cfg.emit("tls_handshake_completed", clientHello) } return &cert.Certificate, err } // getCertificate gets a certificate that matches name from the in-memory // cache, according to the lookup table associated with cfg. The lookup then // points to a certificate in the Instance certificate cache. // // The name is expected to already be normalized (e.g. lowercased). // // If there is no exact match for name, it will be checked against names of // the form '*.example.com' (wildcard certificates) according to RFC 6125. // If a match is found, matched will be true. If no matches are found, matched // will be false and a "default" certificate will be returned with defaulted // set to true. If defaulted is false, then no certificates were available. // // The logic in this function is adapted from the Go standard library, // which is by the Go Authors. // // This function is safe for concurrent use. func (cfg *Config) getCertificate(hello *tls.ClientHelloInfo) (cert Certificate, matched, defaulted bool) { name := normalizedName(hello.ServerName) if name == "" { // if SNI is empty, prefer matching IP address if hello.Conn != nil { addr := localIPFromConn(hello.Conn) cert, matched = cfg.selectCert(hello, addr) if matched { return } } // fall back to a "default" certificate, if specified if cfg.DefaultServerName != "" { normDefault := normalizedName(cfg.DefaultServerName) cert, defaulted = cfg.selectCert(hello, normDefault) if defaulted { return } } } else { // if SNI is specified, try an exact match first cert, matched = cfg.selectCert(hello, name) if matched { return } // try replacing labels in the name with // wildcards until we get a match labels := strings.Split(name, ".") for i := range labels { labels[i] = "*" candidate := strings.Join(labels, ".") cert, matched = cfg.selectCert(hello, candidate) if matched { return } } } // otherwise, we're bingo on ammo; see issues // caddyserver/caddy#2035 and caddyserver/caddy#1303 (any // change to certificate matching behavior must // account for hosts defined where the hostname // is empty or a catch-all, like ":443" or // "0.0.0.0:443") return } // selectCert uses hello to select a certificate from the // cache for name. If cfg.CertSelection is set, it will be // used to make the decision. Otherwise, the first matching // unexpired cert is returned. As a special case, if no // certificates match name and cfg.CertSelection is set, // then all certificates in the cache will be passed in // for the cfg.CertSelection to make the final decision. func (cfg *Config) selectCert(hello *tls.ClientHelloInfo, name string) (Certificate, bool) { logger := loggerNamed(cfg.Logger, "handshake") choices := cfg.certCache.getAllMatchingCerts(name) if len(choices) == 0 { if cfg.CertSelection == nil { if logger != nil { logger.Debug("no matching certificates and no custom selection logic", zap.String("identifier", name)) } return Certificate{}, false } if logger != nil { logger.Debug("no matching certificate; will choose from all certificates", zap.String("identifier", name)) } choices = cfg.certCache.getAllCerts() } if logger != nil { logger.Debug("choosing certificate", zap.String("identifier", name), zap.Int("num_choices", len(choices))) } if cfg.CertSelection == nil { cert, err := DefaultCertificateSelector(hello, choices) if logger != nil { logger.Debug("default certificate selection results", zap.Error(err), zap.String("identifier", name), zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash)) } return cert, err == nil } cert, err := cfg.CertSelection.SelectCertificate(hello, choices) if logger != nil { logger.Debug("custom certificate selection results", zap.Error(err), zap.String("identifier", name), zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.String("issuer_key", cert.issuerKey), zap.String("hash", cert.hash)) } return cert, err == nil } // DefaultCertificateSelector is the default certificate selection logic // given a choice of certificates. If there is at least one certificate in // choices, it always returns a certificate without error. It chooses the // first non-expired certificate that the client supports if possible, // otherwise it returns an expired certificate that the client supports, // otherwise it just returns the first certificate in the list of choices. func DefaultCertificateSelector(hello *tls.ClientHelloInfo, choices []Certificate) (Certificate, error) { if len(choices) == 0 { return Certificate{}, fmt.Errorf("no certificates available") } now := time.Now() best := choices[0] for _, choice := range choices { if err := hello.SupportsCertificate(&choice.Certificate); err != nil { continue } best = choice // at least the client supports it... if now.After(choice.Leaf.NotBefore) && now.Before(choice.Leaf.NotAfter) { return choice, nil // ...and unexpired, great! "Certificate, I choose you!" } } return best, nil // all matching certs are expired or incompatible, oh well } // getCertDuringHandshake will get a certificate for hello. It first tries // the in-memory cache. If no certificate for hello is in the cache, the // config most closely corresponding to hello will be loaded. If that config // allows it (OnDemand==true) and if loadIfNecessary == true, it goes to disk // to load it into the cache and serve it. If it's not on disk and if // obtainIfNecessary == true, the certificate will be obtained from the CA, // cached, and served. If obtainIfNecessary is true, then loadIfNecessary // must also be set to true. An error will be returned if and only if no // certificate is available. // // This function is safe for concurrent use. func (cfg *Config) getCertDuringHandshake(hello *tls.ClientHelloInfo, loadIfNecessary, obtainIfNecessary bool) (Certificate, error) { log := loggerNamed(cfg.Logger, "handshake") // First check our in-memory cache to see if we've already loaded it cert, matched, defaulted := cfg.getCertificate(hello) if matched { if log != nil { log.Debug("matched certificate in cache", zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.Time("expiration", cert.Leaf.NotAfter), zap.String("hash", cert.hash)) } if cert.managed && cfg.OnDemand != nil && obtainIfNecessary { // It's been reported before that if the machine goes to sleep (or // suspends the process) that certs which are already loaded into // memory won't get renewed in the background, so we need to check // expiry on each handshake too, sigh: // https://caddy.community/t/local-certificates-not-renewing-on-demand/9482 return cfg.optionalMaintenance(loggerNamed(cfg.Logger, "on_demand"), cert, hello) } return cert, nil } name := cfg.getNameFromClientHello(hello) // We might be able to load or obtain a needed certificate. Load from // storage if OnDemand is enabled, or if there is the possibility that // a statically-managed cert was evicted from a full cache. cfg.certCache.mu.RLock() cacheSize := len(cfg.certCache.cache) cfg.certCache.mu.RUnlock() // A cert might have still been evicted from the cache even if the cache // is no longer completely full; this happens if the newly-loaded cert is // itself evicted (perhaps due to being expired or unmanaged at this point). // Hence, we use an "almost full" metric to allow for the cache to not be // perfectly full while still being able to load needed certs from storage. // See https://caddy.community/t/error-tls-alert-internal-error-592-again/13272 // and caddyserver/caddy#4320. cacheAlmostFull := float64(cacheSize) >= (float64(cfg.certCache.options.Capacity) * .9) loadDynamically := cfg.OnDemand != nil || cacheAlmostFull if loadDynamically && loadIfNecessary { // Then check to see if we have one on disk // TODO: As suggested here, https://caddy.community/t/error-tls-alert-internal-error-592-again/13272/30?u=matt, // it might be a good idea to check with the DecisionFunc or allowlist first before even loading the certificate // from storage, since if we can't renew it, why should we even try serving it (it will just get evicted after // we get a return value of false anyway)? loadedCert, err := cfg.CacheManagedCertificate(name) if _, ok := err.(ErrNotExist); ok { // If no exact match, try a wildcard variant, which is something we can still use labels := strings.Split(name, ".") labels[0] = "*" loadedCert, err = cfg.CacheManagedCertificate(strings.Join(labels, ".")) } if err == nil { if log != nil { log.Debug("loaded certificate from storage", zap.Strings("subjects", loadedCert.Names), zap.Bool("managed", loadedCert.managed), zap.Time("expiration", loadedCert.Leaf.NotAfter), zap.String("hash", loadedCert.hash)) } loadedCert, err = cfg.handshakeMaintenance(hello, loadedCert) if err != nil { if log != nil { log.Error("maintining newly-loaded certificate", zap.String("server_name", name), zap.Error(err)) } } return loadedCert, nil } if cfg.OnDemand != nil && obtainIfNecessary { // By this point, we need to ask the CA for a certificate return cfg.obtainOnDemandCertificate(hello) } } // Fall back to the default certificate if there is one if defaulted { if log != nil { log.Debug("fell back to default certificate", zap.Strings("subjects", cert.Names), zap.Bool("managed", cert.managed), zap.Time("expiration", cert.Leaf.NotAfter), zap.String("hash", cert.hash)) } return cert, nil } if log != nil { log.Debug("no certificate matching TLS ClientHello", zap.String("server_name", hello.ServerName), zap.String("remote", hello.Conn.RemoteAddr().String()), zap.String("identifier", name), zap.Uint16s("cipher_suites", hello.CipherSuites), zap.Float64("cert_cache_fill", float64(cacheSize)/float64(cfg.certCache.options.Capacity)), // may be approximate! because we are not within the lock zap.Bool("load_if_necessary", loadIfNecessary), zap.Bool("obtain_if_necessary", obtainIfNecessary), zap.Bool("on_demand", cfg.OnDemand != nil)) } return Certificate{}, fmt.Errorf("no certificate available for '%s'", name) } // optionalMaintenance will perform maintenance on the certificate (if necessary) and // will return the resulting certificate. This should only be done if the certificate // is managed, OnDemand is enabled, and the scope is allowed to obtain certificates. func (cfg *Config) optionalMaintenance(log *zap.Logger, cert Certificate, hello *tls.ClientHelloInfo) (Certificate, error) { newCert, err := cfg.handshakeMaintenance(hello, cert) if err == nil { return newCert, nil } if log != nil { log.Error("renewing certificate on-demand failed", zap.Strings("subjects", cert.Names), zap.Time("not_after", cert.Leaf.NotAfter), zap.Error(err)) } if cert.Expired() { return cert, err } // still has time remaining, so serve it anyway return cert, nil } // checkIfCertShouldBeObtained checks to see if an on-demand TLS certificate // should be obtained for a given domain based upon the config settings. If // a non-nil error is returned, do not issue a new certificate for name. func (cfg *Config) checkIfCertShouldBeObtained(name string) error { if cfg.OnDemand == nil { return fmt.Errorf("not configured for on-demand certificate issuance") } if !SubjectQualifiesForCert(name) { return fmt.Errorf("subject name does not qualify for certificate: %s", name) } if cfg.OnDemand.DecisionFunc != nil { return cfg.OnDemand.DecisionFunc(name) } if len(cfg.OnDemand.hostWhitelist) > 0 && !cfg.OnDemand.whitelistContains(name) { return fmt.Errorf("certificate for '%s' is not managed", name) } return nil } // obtainOnDemandCertificate obtains a certificate for hello. // If another goroutine has already started obtaining a cert for // hello, it will wait and use what the other goroutine obtained. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) obtainOnDemandCertificate(hello *tls.ClientHelloInfo) (Certificate, error) { log := loggerNamed(cfg.Logger, "on_demand") name := cfg.getNameFromClientHello(hello) getCertWithoutReobtaining := func() (Certificate, error) { // very important to set the obtainIfNecessary argument to false, so we don't repeat this infinitely return cfg.getCertDuringHandshake(hello, true, false) } // We must protect this process from happening concurrently, so synchronize. obtainCertWaitChansMu.Lock() wait, ok := obtainCertWaitChans[name] if ok { // lucky us -- another goroutine is already obtaining the certificate. // wait for it to finish obtaining the cert and then we'll use it. obtainCertWaitChansMu.Unlock() // TODO: see if we can get a proper context in here, for true cancellation timeout := time.NewTimer(2 * time.Minute) select { case <-timeout.C: return Certificate{}, fmt.Errorf("timed out waiting to obtain certificate for %s", name) case <-wait: timeout.Stop() } return getCertWithoutReobtaining() } // looks like it's up to us to do all the work and obtain the cert. // make a chan others can wait on if needed wait = make(chan struct{}) obtainCertWaitChans[name] = wait obtainCertWaitChansMu.Unlock() unblockWaiters := func() { obtainCertWaitChansMu.Lock() close(wait) delete(obtainCertWaitChans, name) obtainCertWaitChansMu.Unlock() } // Make sure the certificate should be obtained based on config err := cfg.checkIfCertShouldBeObtained(name) if err != nil { unblockWaiters() return Certificate{}, err } if log != nil { log.Info("obtaining new certificate", zap.String("server_name", name)) } // TODO: use a proper context; we use one with timeout because retries are enabled because interactive is false // (timeout duration is based on https://caddy.community/t/zerossl-dns-challenge-failing-often-route53-plugin/13822/24?u=matt) ctx, cancel := context.WithTimeout(context.TODO(), 180*time.Second) defer cancel() // Obtain the certificate err = cfg.ObtainCertAsync(ctx, name) // immediately unblock anyone waiting for it; doing this in // a defer would risk deadlock because of the recursive call // to getCertDuringHandshake below when we return! unblockWaiters() if err != nil { // shucks; failed to solve challenge on-demand return Certificate{}, err } // success; certificate was just placed on disk, so // we need only restart serving the certificate return getCertWithoutReobtaining() } // handshakeMaintenance performs a check on cert for expiration and OCSP validity. // If necessary, it will renew the certificate and/or refresh the OCSP staple. // OCSP stapling errors are not returned, only logged. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) handshakeMaintenance(hello *tls.ClientHelloInfo, cert Certificate) (Certificate, error) { log := loggerNamed(cfg.Logger, "on_demand") // Check cert expiration if currentlyInRenewalWindow(cert.Leaf.NotBefore, cert.Leaf.NotAfter, cfg.RenewalWindowRatio) { return cfg.renewDynamicCertificate(hello, cert) } // Check OCSP staple validity if cert.ocsp != nil { refreshTime := cert.ocsp.ThisUpdate.Add(cert.ocsp.NextUpdate.Sub(cert.ocsp.ThisUpdate) / 2) if time.Now().After(refreshTime) { _, err := stapleOCSP(cfg.OCSP, cfg.Storage, &cert, nil) if err != nil { // An error with OCSP stapling is not the end of the world, and in fact, is // quite common considering not all certs have issuer URLs that support it. if log != nil { log.Warn("stapling OCSP", zap.String("server_name", hello.ServerName), zap.Error(err)) } } cfg.certCache.mu.Lock() cfg.certCache.cache[cert.hash] = cert cfg.certCache.mu.Unlock() } } return cert, nil } // renewDynamicCertificate renews the certificate for name using cfg. It returns the // certificate to use and an error, if any. name should already be lower-cased before // calling this function. name is the name obtained directly from the handshake's // ClientHello. If the certificate hasn't yet expired, currentCert will be returned // and the renewal will happen in the background; otherwise this blocks until the // certificate has been renewed, and returns the renewed certificate. // // This function is safe for use by multiple concurrent goroutines. func (cfg *Config) renewDynamicCertificate(hello *tls.ClientHelloInfo, currentCert Certificate) (Certificate, error) { log := loggerNamed(cfg.Logger, "on_demand") name := cfg.getNameFromClientHello(hello) timeLeft := time.Until(currentCert.Leaf.NotAfter) getCertWithoutReobtaining := func() (Certificate, error) { // very important to set the obtainIfNecessary argument to false, so we don't repeat this infinitely return cfg.getCertDuringHandshake(hello, true, false) } // see if another goroutine is already working on this certificate obtainCertWaitChansMu.Lock() wait, ok := obtainCertWaitChans[name] if ok { // lucky us -- another goroutine is already renewing the certificate obtainCertWaitChansMu.Unlock() if timeLeft > 0 { // the current certificate hasn't expired, and another goroutine is already // renewing it, so we might as well serve what we have without blocking if log != nil { log.Debug("certificate expires soon but is already being renewed; serving current certificate", zap.Strings("subjects", currentCert.Names), zap.Duration("remaining", timeLeft)) } return currentCert, nil } // otherwise, we'll have to wait for the renewal to finish so we don't serve // an expired certificate if log != nil { log.Debug("certificate has expired, but is already being renewed; waiting for renewal to complete", zap.Strings("subjects", currentCert.Names), zap.Time("expired", currentCert.Leaf.NotAfter)) } // TODO: see if we can get a proper context in here, for true cancellation timeout := time.NewTimer(2 * time.Minute) select { case <-timeout.C: return Certificate{}, fmt.Errorf("timed out waiting for certificate renewal of %s", name) case <-wait: timeout.Stop() } return getCertWithoutReobtaining() } // looks like it's up to us to do all the work and renew the cert wait = make(chan struct{}) obtainCertWaitChans[name] = wait obtainCertWaitChansMu.Unlock() unblockWaiters := func() { obtainCertWaitChansMu.Lock() close(wait) delete(obtainCertWaitChans, name) obtainCertWaitChansMu.Unlock() } if log != nil { log.Info("attempting certificate renewal", zap.String("server_name", name), zap.Strings("subjects", currentCert.Names), zap.Time("expiration", currentCert.Leaf.NotAfter), zap.Duration("remaining", timeLeft)) } // Make sure a certificate for this name should be obtained on-demand err := cfg.checkIfCertShouldBeObtained(name) if err != nil { // if not, remove from cache (it will be deleted from storage later) cfg.certCache.mu.Lock() cfg.certCache.removeCertificate(currentCert) cfg.certCache.mu.Unlock() unblockWaiters() return Certificate{}, err } // Renew and reload the certificate renewAndReload := func(ctx context.Context, cancel context.CancelFunc) (Certificate, error) { defer cancel() err = cfg.RenewCertAsync(ctx, name, false) if err == nil { // even though the recursive nature of the dynamic cert loading // would just call this function anyway, we do it here to // make the replacement as atomic as possible. newCert, err := cfg.CacheManagedCertificate(name) if err != nil { if log != nil { log.Error("loading renewed certificate", zap.String("server_name", name), zap.Error(err)) } } else { // replace the old certificate with the new one cfg.certCache.replaceCertificate(currentCert, newCert) } } // immediately unblock anyone waiting for it; doing this in // a defer would risk deadlock because of the recursive call // to getCertDuringHandshake below when we return! unblockWaiters() if err != nil { return Certificate{}, err } return getCertWithoutReobtaining() } // if the certificate hasn't expired, we can serve what we have and renew in the background if timeLeft > 0 { // TODO: get a proper context; we use one with timeout because retries are enabled because interactive is false ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Minute) go renewAndReload(ctx, cancel) return currentCert, nil } // otherwise, we have to block while we renew an expired certificate ctx, cancel := context.WithTimeout(context.TODO(), 90*time.Second) return renewAndReload(ctx, cancel) } // getTLSALPNChallengeCert is to be called when the clientHello pertains to // a TLS-ALPN challenge and a certificate is required to solve it. This method gets // the relevant challenge info and then returns the associated certificate (if any) // or generates it anew if it's not available (as is the case when distributed // solving). True is returned if the challenge is being solved distributed (there // is no semantic difference with distributed solving; it is mainly for logging). func (cfg *Config) getTLSALPNChallengeCert(clientHello *tls.ClientHelloInfo) (*tls.Certificate, bool, error) { chalData, distributed, err := cfg.getChallengeInfo(clientHello.ServerName) if err != nil { return nil, distributed, err } // fast path: we already created the certificate (this avoids having to re-create // it at every handshake that tries to verify, e.g. multi-perspective validation) if chalData.data != nil { return chalData.data.(*tls.Certificate), distributed, nil } // otherwise, we can re-create the solution certificate, but it takes a few cycles cert, err := acmez.TLSALPN01ChallengeCert(chalData.Challenge) if err != nil { return nil, distributed, fmt.Errorf("making TLS-ALPN challenge certificate: %v", err) } if cert == nil { return nil, distributed, fmt.Errorf("got nil TLS-ALPN challenge certificate but no error") } return cert, distributed, nil } // getNameFromClientHello returns a normalized form of hello.ServerName. // If hello.ServerName is empty (i.e. client did not use SNI), then the // associated connection's local address is used to extract an IP address. func (*Config) getNameFromClientHello(hello *tls.ClientHelloInfo) string { if name := normalizedName(hello.ServerName); name != "" { return name } return localIPFromConn(hello.Conn) } // localIPFromConn returns the host portion of c's local address // and strips the scope ID if one exists (see RFC 4007). func localIPFromConn(c net.Conn) string { if c == nil { return "" } localAddr := c.LocalAddr().String() ip, _, err := net.SplitHostPort(localAddr) if err != nil { // OK; assume there was no port ip = localAddr } // IPv6 addresses can have scope IDs, e.g. "fe80::4c3:3cff:fe4f:7e0b%eth0", // but for our purposes, these are useless (unless a valid use case proves // otherwise; see issue #3911) if scopeIDStart := strings.Index(ip, "%"); scopeIDStart > -1 { ip = ip[:scopeIDStart] } return ip } // normalizedName returns a cleaned form of serverName that is // used for consistency when referring to a SNI value. func normalizedName(serverName string) string { return strings.ToLower(strings.TrimSpace(serverName)) } // obtainCertWaitChans is used to coordinate obtaining certs for each hostname. var obtainCertWaitChans = make(map[string]chan struct{}) var obtainCertWaitChansMu sync.Mutex golang-github-caddyserver-certmagic-0.15.2/handshake_test.go000066400000000000000000000103161414454377300241240ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "crypto/tls" "crypto/x509" "net" "testing" ) func TestGetCertificate(t *testing.T) { c := &Cache{ cache: make(map[string]Certificate), cacheIndex: make(map[string][]string), } cfg := &Config{certCache: c} // create a test connection for conn.LocalAddr() l, _ := net.Listen("tcp", "127.0.0.1:0") defer l.Close() conn, _ := net.Dial("tcp", l.Addr().String()) if conn == nil { t.Errorf("failed to create a test connection") } defer conn.Close() hello := &tls.ClientHelloInfo{ServerName: "example.com", Conn: conn} helloSub := &tls.ClientHelloInfo{ServerName: "sub.example.com", Conn: conn} helloNoSNI := &tls.ClientHelloInfo{Conn: conn} helloNoMatch := &tls.ClientHelloInfo{ServerName: "nomatch", Conn: conn} // When cache is empty if cert, err := cfg.GetCertificate(hello); err == nil { t.Errorf("GetCertificate should return error when cache is empty, got: %v", cert) } if cert, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("GetCertificate should return error when cache is empty even if server name is blank, got: %v", cert) } // When cache has one certificate in it firstCert := Certificate{Names: []string{"example.com"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{DNSNames: []string{"example.com"}}}} c.cacheCertificate(firstCert) if cert, err := cfg.GetCertificate(hello); err != nil { t.Errorf("Got an error but shouldn't have, when cert exists in cache: %v", err) } else if cert.Leaf.DNSNames[0] != "example.com" { t.Errorf("Got wrong certificate with exact match; expected 'example.com', got: %v", cert) } if _, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("Did not get an error with no SNI and no DefaultServerName, but should have: %v", err) } // When retrieving wildcard certificate wildcardCert := Certificate{ Names: []string{"*.example.com"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{DNSNames: []string{"*.example.com"}}}, hash: "(don't overwrite the first one)", } c.cacheCertificate(wildcardCert) if cert, err := cfg.GetCertificate(helloSub); err != nil { t.Errorf("Didn't get wildcard cert, got: cert=%v, err=%v ", cert, err) } else if cert.Leaf.DNSNames[0] != "*.example.com" { t.Errorf("Got wrong certificate, expected wildcard: %v", cert) } // When cache is NOT empty but there's no SNI if _, err := cfg.GetCertificate(helloNoSNI); err == nil { t.Errorf("Expected TLS allert when no SNI and no DefaultServerName, but got: %v", err) } // When no certificate matches, raise an alert if _, err := cfg.GetCertificate(helloNoMatch); err == nil { t.Errorf("Expected an error when no certificate matched the SNI, got: %v", err) } // When default SNI is set and SNI is missing, retrieve default cert cfg.DefaultServerName = "example.com" if cert, err := cfg.GetCertificate(helloNoSNI); err != nil { t.Errorf("Got an error with no SNI with DefaultServerName, but shouldn't have: %v", err) } else if cert == nil || cert.Leaf.DNSNames[0] != "example.com" { t.Errorf("Expected default cert, got: %v", cert) } // When default SNI is set and SNI is missing but IP address matches, retrieve IP cert ipCert := Certificate{ Names: []string{"127.0.0.1"}, Certificate: tls.Certificate{Leaf: &x509.Certificate{IPAddresses: []net.IP{net.ParseIP("127.0.0.1")}}}, hash: "(don't overwrite the first or second one)", } c.cacheCertificate(ipCert) if cert, err := cfg.GetCertificate(helloNoSNI); err != nil { t.Errorf("Got an error with no SNI but matching IP, but shouldn't have: %v", err) } else if cert == nil || len(cert.Leaf.IPAddresses) == 0 { t.Errorf("Expected IP cert, got: %v", cert) } } golang-github-caddyserver-certmagic-0.15.2/httphandler.go000066400000000000000000000113401414454377300234520ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "net/http" "strings" "github.com/mholt/acmez/acme" "go.uber.org/zap" ) // HTTPChallengeHandler wraps h in a handler that can solve the ACME // HTTP challenge. cfg is required, and it must have a certificate // cache backed by a functional storage facility, since that is where // the challenge state is stored between initiation and solution. // // If a request is not an ACME HTTP challenge, h will be invoked. func (am *ACMEManager) HTTPChallengeHandler(h http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { if am.HandleHTTPChallenge(w, r) { return } h.ServeHTTP(w, r) }) } // HandleHTTPChallenge uses am to solve challenge requests from an ACME // server that were initiated by this instance or any other instance in // this cluster (being, any instances using the same storage am does). // // If the HTTP challenge is disabled, this function is a no-op. // // If am is nil or if am does not have a certificate cache backed by // usable storage, solving the HTTP challenge will fail. // // It returns true if it handled the request; if so, the response has // already been written. If false is returned, this call was a no-op and // the request has not been handled. func (am *ACMEManager) HandleHTTPChallenge(w http.ResponseWriter, r *http.Request) bool { if am == nil { return false } if am.DisableHTTPChallenge { return false } if !LooksLikeHTTPChallenge(r) { return false } return am.distributedHTTPChallengeSolver(w, r) } // distributedHTTPChallengeSolver checks to see if this challenge // request was initiated by this or another instance which uses the // same storage as am does, and attempts to complete the challenge for // it. It returns true if the request was handled; false otherwise. func (am *ACMEManager) distributedHTTPChallengeSolver(w http.ResponseWriter, r *http.Request) bool { if am == nil { return false } host := hostOnly(r.Host) chalInfo, distributed, err := am.config.getChallengeInfo(host) if err != nil { if am.Logger != nil { am.Logger.Error("looking up info for HTTP challenge", zap.String("host", host), zap.Error(err)) } return false } return solveHTTPChallenge(am.Logger, w, r, chalInfo.Challenge, distributed) } // solveHTTPChallenge solves the HTTP challenge using the given challenge information. // If the challenge is being solved in a distributed fahsion, set distributed to true for logging purposes. // It returns true the properties of the request check out in relation to the HTTP challenge. // Most of this code borrowed from xenolf's built-in HTTP-01 challenge solver in March 2018. func solveHTTPChallenge(logger *zap.Logger, w http.ResponseWriter, r *http.Request, challenge acme.Challenge, distributed bool) bool { challengeReqPath := challenge.HTTP01ResourcePath() if r.URL.Path == challengeReqPath && strings.EqualFold(hostOnly(r.Host), challenge.Identifier.Value) && // mitigate DNS rebinding attacks r.Method == "GET" { w.Header().Add("Content-Type", "text/plain") w.Write([]byte(challenge.KeyAuthorization)) r.Close = true if logger != nil { logger.Info("served key authentication", zap.String("identifier", challenge.Identifier.Value), zap.String("challenge", "http-01"), zap.String("remote", r.RemoteAddr), zap.Bool("distributed", distributed)) } return true } return false } // SolveHTTPChallenge solves the HTTP challenge. It should be used only on HTTP requests that are // from ACME servers trying to validate an identifier (i.e. LooksLikeHTTPChallenge() == true). It // returns true if the request criteria check out and it answered with key authentication, in which // case no further handling of the request is necessary. func SolveHTTPChallenge(logger *zap.Logger, w http.ResponseWriter, r *http.Request, challenge acme.Challenge) bool { return solveHTTPChallenge(logger, w, r, challenge, false) } // LooksLikeHTTPChallenge returns true if r looks like an ACME // HTTP challenge request from an ACME server. func LooksLikeHTTPChallenge(r *http.Request) bool { return r.Method == "GET" && strings.HasPrefix(r.URL.Path, challengeBasePath) } const challengeBasePath = "/.well-known/acme-challenge" golang-github-caddyserver-certmagic-0.15.2/httphandler_test.go000066400000000000000000000033511414454377300245140ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "net/http" "net/http/httptest" "os" "testing" ) func TestHTTPChallengeHandlerNoOp(t *testing.T) { am := &ACMEManager{CA: "https://example.com/acme/directory"} testConfig := &Config{ Issuers: []Issuer{am}, Storage: &FileStorage{Path: "./_testdata_tmp"}, certCache: new(Cache), } am.config = testConfig testStorageDir := testConfig.Storage.(*FileStorage).Path defer func() { err := os.RemoveAll(testStorageDir) if err != nil { t.Fatalf("Could not remove temporary storage directory (%s): %v", testStorageDir, err) } }() // try base paths and host names that aren't // handled by this handler for _, url := range []string{ "http://localhost/", "http://localhost/foo.html", "http://localhost/.git", "http://localhost/.well-known/", "http://localhost/.well-known/acme-challenging", "http://other/.well-known/acme-challenge/foo", } { req, err := http.NewRequest("GET", url, nil) if err != nil { t.Fatalf("Could not craft request, got error: %v", err) } rw := httptest.NewRecorder() if am.HandleHTTPChallenge(rw, req) { t.Errorf("Got true with this URL, but shouldn't have: %s", url) } } } golang-github-caddyserver-certmagic-0.15.2/maintain.go000066400000000000000000000513401414454377300227410ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/x509" "encoding/pem" "fmt" "log" "path" "runtime" "strings" "time" "github.com/mholt/acmez/acme" "go.uber.org/zap" "golang.org/x/crypto/ocsp" ) // maintainAssets is a permanently-blocking function // that loops indefinitely and, on a regular schedule, checks // certificates for expiration and initiates a renewal of certs // that are expiring soon. It also updates OCSP stapling. It // should only be called once per cache. Panics are recovered, // and if panicCount < 10, the function is called recursively, // incrementing panicCount each time. Initial invocation should // start panicCount at 0. func (certCache *Cache) maintainAssets(panicCount int) { log := loggerNamed(certCache.logger, "maintenance") if log != nil { log = log.With(zap.String("cache", fmt.Sprintf("%p", certCache))) } defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] if log != nil { log.Error("panic", zap.Any("error", err), zap.ByteString("stack", buf)) } if panicCount < 10 { certCache.maintainAssets(panicCount + 1) } } }() renewalTicker := time.NewTicker(certCache.options.RenewCheckInterval) ocspTicker := time.NewTicker(certCache.options.OCSPCheckInterval) if log != nil { log.Info("started background certificate maintenance") } ctx, cancel := context.WithCancel(context.Background()) defer cancel() for { select { case <-renewalTicker.C: err := certCache.RenewManagedCertificates(ctx) if err != nil && log != nil { log.Error("renewing managed certificates", zap.Error(err)) } case <-ocspTicker.C: certCache.updateOCSPStaples(ctx) case <-certCache.stopChan: renewalTicker.Stop() ocspTicker.Stop() // TODO: stop any in-progress maintenance operations and clear locks we made (this might be done now with our use of context) if log != nil { log.Info("stopped background certificate maintenance") } close(certCache.doneChan) return } } } // RenewManagedCertificates renews managed certificates, // including ones loaded on-demand. Note that this is done // automatically on a regular basis; normally you will not // need to call this. This method assumes non-interactive // mode (i.e. operating in the background). func (certCache *Cache) RenewManagedCertificates(ctx context.Context) error { log := loggerNamed(certCache.logger, "maintenance") // configs will hold a map of certificate name to the config // to use when managing that certificate configs := make(map[string]*Config) // we use the queues for a very important reason: to do any and all // operations that could require an exclusive write lock outside // of the read lock! otherwise we get a deadlock, yikes. in other // words, our first iteration through the certificate cache does NOT // perform any operations--only queues them--so that more fine-grained // write locks may be obtained during the actual operations. var renewQueue, reloadQueue, deleteQueue []Certificate certCache.mu.RLock() for certKey, cert := range certCache.cache { if !cert.managed { continue } // the list of names on this cert should never be empty... programmer error? if cert.Names == nil || len(cert.Names) == 0 { if log != nil { log.Warn("certificate has no names; removing from cache", zap.String("cert_key", certKey)) } deleteQueue = append(deleteQueue, cert) continue } // get the config associated with this certificate cfg, err := certCache.getConfig(cert) if err != nil { if log != nil { log.Error("unable to get configuration to manage certificate; unable to renew", zap.Strings("identifiers", cert.Names), zap.Error(err)) } continue } if cfg == nil { // this is bad if this happens, probably a programmer error (oops) if log != nil { log.Error("no configuration associated with certificate; unable to manage", zap.Strings("identifiers", cert.Names)) } continue } if cfg.OnDemand != nil { continue } // if time is up or expires soon, we need to try to renew it if cert.NeedsRenewal(cfg) { configs[cert.Names[0]] = cfg // see if the certificate in storage has already been renewed, possibly by another // instance that didn't coordinate with this one; if so, just load it (this // might happen if another instance already renewed it - kinda sloppy but checking disk // first is a simple way to possibly drastically reduce rate limit problems) storedCertExpiring, err := cfg.managedCertInStorageExpiresSoon(cert) if err != nil { // hmm, weird, but not a big deal, maybe it was deleted or something if log != nil { log.Warn("error while checking if stored certificate is also expiring soon", zap.Strings("identifiers", cert.Names), zap.Error(err)) } } else if !storedCertExpiring { // if the certificate is NOT expiring soon and there was no error, then we // are good to just reload the certificate from storage instead of repeating // a likely-unnecessary renewal procedure reloadQueue = append(reloadQueue, cert) continue } // the certificate in storage has not been renewed yet, so we will do it // NOTE: It is super-important to note that the TLS-ALPN challenge requires // a write lock on the cache in order to complete its challenge, so it is extra // vital that this renew operation does not happen inside our read lock! renewQueue = append(renewQueue, cert) } } certCache.mu.RUnlock() // Reload certificates that merely need to be updated in memory for _, oldCert := range reloadQueue { timeLeft := oldCert.Leaf.NotAfter.Sub(time.Now().UTC()) if log != nil { log.Info("certificate expires soon, but is already renewed in storage; reloading stored certificate", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) } cfg := configs[oldCert.Names[0]] // crucially, this happens OUTSIDE a lock on the certCache err := cfg.reloadManagedCertificate(oldCert) if err != nil { if log != nil { log.Error("loading renewed certificate", zap.Strings("identifiers", oldCert.Names), zap.Error(err)) } continue } } // Renewal queue for _, oldCert := range renewQueue { cfg := configs[oldCert.Names[0]] err := certCache.queueRenewalTask(ctx, oldCert, cfg) if err != nil { if log != nil { log.Error("queueing renewal task", zap.Strings("identifiers", oldCert.Names), zap.Error(err)) } continue } } // Deletion queue certCache.mu.Lock() for _, cert := range deleteQueue { certCache.removeCertificate(cert) } certCache.mu.Unlock() return nil } func (certCache *Cache) queueRenewalTask(ctx context.Context, oldCert Certificate, cfg *Config) error { log := loggerNamed(certCache.logger, "maintenance") timeLeft := oldCert.Leaf.NotAfter.Sub(time.Now().UTC()) if log != nil { log.Info("certificate expires soon; queuing for renewal", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) } // Get the name which we should use to renew this certificate; // we only support managing certificates with one name per cert, // so this should be easy. renewName := oldCert.Names[0] // queue up this renewal job (is a no-op if already active or queued) jm.Submit(cfg.Logger, "renew_"+renewName, func() error { timeLeft := oldCert.Leaf.NotAfter.Sub(time.Now().UTC()) if log != nil { log.Info("attempting certificate renewal", zap.Strings("identifiers", oldCert.Names), zap.Duration("remaining", timeLeft)) } // perform renewal - crucially, this happens OUTSIDE a lock on certCache err := cfg.RenewCertAsync(ctx, renewName, false) if err != nil { if cfg.OnDemand != nil { // loaded dynamically, remove dynamically certCache.mu.Lock() certCache.removeCertificate(oldCert) certCache.mu.Unlock() } return fmt.Errorf("%v %v", oldCert.Names, err) } // successful renewal, so update in-memory cache by loading // renewed certificate so it will be used with handshakes err = cfg.reloadManagedCertificate(oldCert) if err != nil { return ErrNoRetry{fmt.Errorf("%v %v", oldCert.Names, err)} } return nil }) return nil } // updateOCSPStaples updates the OCSP stapling in all // eligible, cached certificates. // // OCSP maintenance strives to abide the relevant points on // Ryan Sleevi's recommendations for good OCSP support: // https://gist.github.com/sleevi/5efe9ef98961ecfb4da8 func (certCache *Cache) updateOCSPStaples(ctx context.Context) { logger := loggerNamed(certCache.logger, "maintenance") // temporary structures to store updates or tasks // so that we can keep our locks short-lived type ocspUpdate struct { rawBytes []byte parsed *ocsp.Response } type updateQueueEntry struct { cert Certificate certHash string lastNextUpdate time.Time } type renewQueueEntry struct { oldCert Certificate ocspResp *ocsp.Response } updated := make(map[string]ocspUpdate) var updateQueue []updateQueueEntry // certs that need a refreshed staple var renewQueue []renewQueueEntry // certs that need to be renewed (due to revocation) configs := make(map[string]*Config) // obtain brief read lock during our scan to see which staples need updating certCache.mu.RLock() for certHash, cert := range certCache.cache { // no point in updating OCSP for expired or "synthetic" certificates if cert.Leaf == nil || cert.Expired() { continue } var lastNextUpdate time.Time if cert.ocsp != nil { lastNextUpdate = cert.ocsp.NextUpdate if freshOCSP(cert.ocsp) { continue // no need to update staple if ours is still fresh } } updateQueue = append(updateQueue, updateQueueEntry{cert, certHash, lastNextUpdate}) } certCache.mu.RUnlock() // perform updates outside of any lock on certCache for _, qe := range updateQueue { cert := qe.cert certHash := qe.certHash lastNextUpdate := qe.lastNextUpdate cfg, err := certCache.getConfig(cert) if err != nil { if logger != nil { logger.Error("unable to refresh OCSP staple because getting automation config for certificate failed", zap.Strings("identifiers", cert.Names), zap.Error(err)) } continue } if cfg == nil { // this is bad if this happens, probably a programmer error (oops) if logger != nil { logger.Error("no configuration associated with certificate; unable to manage OCSP staples", zap.Strings("identifiers", cert.Names)) } continue } ocspResp, err := stapleOCSP(cfg.OCSP, cfg.Storage, &cert, nil) if err != nil || ocspResp == nil { if cert.ocsp != nil { // if there was no staple before, that's fine; otherwise we should log the error if logger != nil { logger.Error("stapling OCSP", zap.Strings("identifiers", cert.Names), zap.Error(err)) } } continue } // By this point, we've obtained the latest OCSP response. // If there was no staple before, or if the response is updated, make // sure we apply the update to all names on the certificate. if cert.ocsp != nil && (lastNextUpdate.IsZero() || lastNextUpdate != cert.ocsp.NextUpdate) { if logger != nil { logger.Info("advancing OCSP staple", zap.Strings("identifiers", cert.Names), zap.Time("from", lastNextUpdate), zap.Time("to", cert.ocsp.NextUpdate)) } updated[certHash] = ocspUpdate{rawBytes: cert.Certificate.OCSPStaple, parsed: cert.ocsp} } // If a managed certificate was revoked, we should attempt to replace it with a new one. if cert.managed && ocspResp.Status == ocsp.Revoked && len(cert.Names) > 0 { renewQueue = append(renewQueue, renewQueueEntry{ oldCert: cert, ocspResp: ocspResp, }) configs[cert.Names[0]] = cfg } } // These write locks should be brief since we have all the info we need now. for certKey, update := range updated { certCache.mu.Lock() cert := certCache.cache[certKey] cert.ocsp = update.parsed cert.Certificate.OCSPStaple = update.rawBytes certCache.cache[certKey] = cert certCache.mu.Unlock() } // We attempt to replace any certificates that were revoked. // Crucially, this happens OUTSIDE a lock on the certCache. for _, renew := range renewQueue { if logger != nil { logger.Warn("OCSP status for managed certificate is REVOKED; attempting to replace with new certificate", zap.Strings("identifiers", renew.oldCert.Names), zap.Time("expiration", renew.oldCert.Leaf.NotAfter)) } renewName := renew.oldCert.Names[0] cfg := configs[renewName] // if revoked for key compromise, we can't be sure whether the storage of // the key is still safe; however, we KNOW the old key is not safe, and we // can only hope by the time of revocation that storage has been secured; // key management is not something we want to get into, but in this case // it seems prudent to replace the key - and since renewal requires reuse // of a prior key, we can't do a "renew" to replace the cert if we need a // new key, so we'll have to do an obtain instead var obtainInsteadOfRenew bool if renew.ocspResp.RevocationReason == acme.ReasonKeyCompromise { err := cfg.moveCompromisedPrivateKey(renew.oldCert, logger) if err != nil && logger != nil { logger.Error("could not remove compromised private key from use", zap.Strings("identifiers", renew.oldCert.Names), zap.String("issuer", renew.oldCert.issuerKey), zap.Error(err)) } obtainInsteadOfRenew = true } var err error if obtainInsteadOfRenew { err = cfg.ObtainCertAsync(ctx, renewName) } else { // notice that we force renewal; otherwise, it might see that the // certificate isn't close to expiring and return, but we really // need a replacement certificate! see issue #4191 err = cfg.RenewCertAsync(ctx, renewName, true) } if err != nil { // probably better to not serve a revoked certificate at all if logger != nil { logger.Error("unable to obtain new to certificate after OCSP status of REVOKED; removing from cache", zap.Strings("identifiers", renew.oldCert.Names), zap.Error(err)) } certCache.mu.Lock() certCache.removeCertificate(renew.oldCert) certCache.mu.Unlock() continue } err = cfg.reloadManagedCertificate(renew.oldCert) if err != nil { if logger != nil { logger.Error("after obtaining new certificate due to OCSP status of REVOKED", zap.Strings("identifiers", renew.oldCert.Names), zap.Error(err)) } continue } } } // CleanStorageOptions specifies how to clean up a storage unit. type CleanStorageOptions struct { OCSPStaples bool ExpiredCerts bool ExpiredCertGracePeriod time.Duration } // CleanStorage removes assets which are no longer useful, // according to opts. func CleanStorage(ctx context.Context, storage Storage, opts CleanStorageOptions) { if opts.OCSPStaples { err := deleteOldOCSPStaples(ctx, storage) if err != nil { log.Printf("[ERROR] Deleting old OCSP staples: %v", err) } } if opts.ExpiredCerts { err := deleteExpiredCerts(ctx, storage, opts.ExpiredCertGracePeriod) if err != nil { log.Printf("[ERROR] Deleting expired certificates: %v", err) } } // TODO: delete stale locks? } func deleteOldOCSPStaples(ctx context.Context, storage Storage) error { ocspKeys, err := storage.List(prefixOCSP, false) if err != nil { // maybe just hasn't been created yet; no big deal return nil } for _, key := range ocspKeys { // if context was cancelled, quit early; otherwise proceed select { case <-ctx.Done(): return ctx.Err() default: } ocspBytes, err := storage.Load(key) if err != nil { log.Printf("[ERROR] While deleting old OCSP staples, unable to load staple file: %v", err) continue } resp, err := ocsp.ParseResponse(ocspBytes, nil) if err != nil { // contents are invalid; delete it err = storage.Delete(key) if err != nil { log.Printf("[ERROR] Purging corrupt staple file %s: %v", key, err) } continue } if time.Now().After(resp.NextUpdate) { // response has expired; delete it err = storage.Delete(key) if err != nil { log.Printf("[ERROR] Purging expired staple file %s: %v", key, err) } } } return nil } func deleteExpiredCerts(ctx context.Context, storage Storage, gracePeriod time.Duration) error { issuerKeys, err := storage.List(prefixCerts, false) if err != nil { // maybe just hasn't been created yet; no big deal return nil } for _, issuerKey := range issuerKeys { siteKeys, err := storage.List(issuerKey, false) if err != nil { log.Printf("[ERROR] Listing contents of %s: %v", issuerKey, err) continue } for _, siteKey := range siteKeys { // if context was cancelled, quit early; otherwise proceed select { case <-ctx.Done(): return ctx.Err() default: } siteAssets, err := storage.List(siteKey, false) if err != nil { log.Printf("[ERROR] Listing contents of %s: %v", siteKey, err) continue } for _, assetKey := range siteAssets { if path.Ext(assetKey) != ".crt" { continue } certFile, err := storage.Load(assetKey) if err != nil { return fmt.Errorf("loading certificate file %s: %v", assetKey, err) } block, _ := pem.Decode(certFile) if block == nil || block.Type != "CERTIFICATE" { return fmt.Errorf("certificate file %s does not contain PEM-encoded certificate", assetKey) } cert, err := x509.ParseCertificate(block.Bytes) if err != nil { return fmt.Errorf("certificate file %s is malformed; error parsing PEM: %v", assetKey, err) } if expiredTime := time.Since(cert.NotAfter); expiredTime >= gracePeriod { log.Printf("[INFO] Certificate %s expired %s ago; cleaning up", assetKey, expiredTime) baseName := strings.TrimSuffix(assetKey, ".crt") for _, relatedAsset := range []string{ assetKey, baseName + ".key", baseName + ".json", } { log.Printf("[INFO] Deleting %s because resource expired", relatedAsset) err := storage.Delete(relatedAsset) if err != nil { log.Printf("[ERROR] Cleaning up asset related to expired certificate for %s: %s: %v", baseName, relatedAsset, err) } } } } // update listing; if folder is empty, delete it siteAssets, err = storage.List(siteKey, false) if err != nil { continue } if len(siteAssets) == 0 { log.Printf("[INFO] Deleting %s because key is empty", siteKey) err := storage.Delete(siteKey) if err != nil { return fmt.Errorf("deleting empty site folder %s: %v", siteKey, err) } } } } return nil } // moveCompromisedPrivateKey moves the private key for cert to a ".compromised" file // by copying the data to the new file, then deleting the old one. func (cfg *Config) moveCompromisedPrivateKey(cert Certificate, logger *zap.Logger) error { privKeyStorageKey := StorageKeys.SitePrivateKey(cert.issuerKey, cert.Names[0]) privKeyPEM, err := cfg.Storage.Load(privKeyStorageKey) if err != nil { return err } compromisedPrivKeyStorageKey := privKeyStorageKey + ".compromised" err = cfg.Storage.Store(compromisedPrivKeyStorageKey, privKeyPEM) if err != nil { // better safe than sorry: as a last resort, try deleting the key so it won't be reused cfg.Storage.Delete(privKeyStorageKey) return err } err = cfg.Storage.Delete(privKeyStorageKey) if err != nil { return err } logger.Info("removed certificate's compromised private key from use", zap.String("storage_path", compromisedPrivKeyStorageKey), zap.Strings("identifiers", cert.Names), zap.String("issuer", cert.issuerKey)) return nil } const ( // DefaultRenewCheckInterval is how often to check certificates for expiration. // Scans are very lightweight, so this can be semi-frequent. This default should // be smaller than *DefaultRenewalWindowRatio/3, which // gives certificates plenty of chance to be renewed on time. DefaultRenewCheckInterval = 10 * time.Minute // DefaultRenewalWindowRatio is how much of a certificate's lifetime becomes the // renewal window. The renewal window is the span of time at the end of the // certificate's validity period in which it should be renewed. A default value // of ~1/3 is pretty safe and recommended for most certificates. DefaultRenewalWindowRatio = 1.0 / 3.0 // DefaultOCSPCheckInterval is how often to check if OCSP stapling needs updating. DefaultOCSPCheckInterval = 1 * time.Hour ) golang-github-caddyserver-certmagic-0.15.2/ocsp.go000066400000000000000000000201221414454377300220770ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "bytes" "crypto/x509" "encoding/pem" "fmt" "io" "io/ioutil" "log" "net/http" "time" "golang.org/x/crypto/ocsp" ) // stapleOCSP staples OCSP information to cert for hostname name. // If you have it handy, you should pass in the PEM-encoded certificate // bundle; otherwise the DER-encoded cert will have to be PEM-encoded. // If you don't have the PEM blocks already, just pass in nil. // // Errors here are not necessarily fatal, it could just be that the // certificate doesn't have an issuer URL. This function may return // both nil values if OCSP stapling is disabled according to ocspConfig. // // If a status was received, it returns that status. Note that the // returned status is not always stapled to the certificate. func stapleOCSP(ocspConfig OCSPConfig, storage Storage, cert *Certificate, pemBundle []byte) (*ocsp.Response, error) { if ocspConfig.DisableStapling { return nil, nil } if pemBundle == nil { // we need a PEM encoding only for some function calls below bundle := new(bytes.Buffer) for _, derBytes := range cert.Certificate.Certificate { pem.Encode(bundle, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes}) } pemBundle = bundle.Bytes() } var ocspBytes []byte var ocspResp *ocsp.Response var ocspErr error var gotNewOCSP bool // First try to load OCSP staple from storage and see if // we can still use it. ocspStapleKey := StorageKeys.OCSPStaple(cert, pemBundle) cachedOCSP, err := storage.Load(ocspStapleKey) if err == nil { resp, err := ocsp.ParseResponse(cachedOCSP, nil) if err == nil { if freshOCSP(resp) { // staple is still fresh; use it ocspBytes = cachedOCSP ocspResp = resp } } else { // invalid contents; delete the file // (we do this independently of the maintenance routine because // in this case we know for sure this should be a staple file // because we loaded it by name, whereas the maintenance routine // just iterates the list of files, even if somehow a non-staple // file gets in the folder. in this case we are sure it is corrupt.) err := storage.Delete(ocspStapleKey) if err != nil { log.Printf("[WARNING] Unable to delete invalid OCSP staple file: %v", err) } } } // If we couldn't get a fresh staple by reading the cache, // then we need to request it from the OCSP responder if ocspResp == nil || len(ocspBytes) == 0 { ocspBytes, ocspResp, ocspErr = getOCSPForCert(ocspConfig, pemBundle) if ocspErr != nil { // An error here is not a problem because a certificate may simply // not contain a link to an OCSP server. But we should log it anyway. // There's nothing else we can do to get OCSP for this certificate, // so we can return here with the error. return nil, fmt.Errorf("no OCSP stapling for %v: %v", cert.Names, ocspErr) } gotNewOCSP = true } // By now, we should have a response. If good, staple it to // the certificate. If the OCSP response was not loaded from // storage, we persist it for next time. if ocspResp.Status == ocsp.Good { if ocspResp.NextUpdate.After(cert.Leaf.NotAfter) { // uh oh, this OCSP response expires AFTER the certificate does, that's kinda bogus. // it was the reason a lot of Symantec-validated sites (not Caddy) went down // in October 2017. https://twitter.com/mattiasgeniar/status/919432824708648961 return ocspResp, fmt.Errorf("invalid: OCSP response for %v valid after certificate expiration (%s)", cert.Names, cert.Leaf.NotAfter.Sub(ocspResp.NextUpdate)) } cert.Certificate.OCSPStaple = ocspBytes cert.ocsp = ocspResp if gotNewOCSP { err := storage.Store(ocspStapleKey, ocspBytes) if err != nil { return ocspResp, fmt.Errorf("unable to write OCSP staple file for %v: %v", cert.Names, err) } } } return ocspResp, nil } // getOCSPForCert takes a PEM encoded cert or cert bundle returning the raw OCSP response, // the parsed response, and an error, if any. The returned []byte can be passed directly // into the OCSPStaple property of a tls.Certificate. If the bundle only contains the // issued certificate, this function will try to get the issuer certificate from the // IssuingCertificateURL in the certificate. If the []byte and/or ocsp.Response return // values are nil, the OCSP status may be assumed OCSPUnknown. // // Borrowed from xenolf. func getOCSPForCert(ocspConfig OCSPConfig, bundle []byte) ([]byte, *ocsp.Response, error) { // TODO: Perhaps this should be synchronized too, with a Locker? certificates, err := parseCertsFromPEMBundle(bundle) if err != nil { return nil, nil, err } // We expect the certificate slice to be ordered downwards the chain. // SRV CRT -> CA. We need to pull the leaf and issuer certs out of it, // which should always be the first two certificates. If there's no // OCSP server listed in the leaf cert, there's nothing to do. And if // we have only one certificate so far, we need to get the issuer cert. issuedCert := certificates[0] if len(issuedCert.OCSPServer) == 0 { return nil, nil, fmt.Errorf("no OCSP server specified in certificate") } // apply override for responder URL respURL := issuedCert.OCSPServer[0] if len(ocspConfig.ResponderOverrides) > 0 { if override, ok := ocspConfig.ResponderOverrides[respURL]; ok { respURL = override } } if respURL == "" { return nil, nil, fmt.Errorf("override disables querying OCSP responder: %v", issuedCert.OCSPServer[0]) } if len(certificates) == 1 { if len(issuedCert.IssuingCertificateURL) == 0 { return nil, nil, fmt.Errorf("no URL to issuing certificate") } resp, err := http.Get(issuedCert.IssuingCertificateURL[0]) if err != nil { return nil, nil, fmt.Errorf("getting issuer certificate: %v", err) } defer resp.Body.Close() issuerBytes, err := ioutil.ReadAll(io.LimitReader(resp.Body, 1024*1024)) if err != nil { return nil, nil, fmt.Errorf("reading issuer certificate: %v", err) } issuerCert, err := x509.ParseCertificate(issuerBytes) if err != nil { return nil, nil, fmt.Errorf("parsing issuer certificate: %v", err) } // insert it into the slice on position 0; // we want it ordered right SRV CRT -> CA certificates = append(certificates, issuerCert) } issuerCert := certificates[1] ocspReq, err := ocsp.CreateRequest(issuedCert, issuerCert, nil) if err != nil { return nil, nil, fmt.Errorf("creating OCSP request: %v", err) } reader := bytes.NewReader(ocspReq) req, err := http.Post(respURL, "application/ocsp-request", reader) if err != nil { return nil, nil, fmt.Errorf("making OCSP request: %v", err) } defer req.Body.Close() ocspResBytes, err := ioutil.ReadAll(io.LimitReader(req.Body, 1024*1024)) if err != nil { return nil, nil, fmt.Errorf("reading OCSP response: %v", err) } ocspRes, err := ocsp.ParseResponse(ocspResBytes, issuerCert) if err != nil { return nil, nil, fmt.Errorf("parsing OCSP response: %v", err) } return ocspResBytes, ocspRes, nil } // freshOCSP returns true if resp is still fresh, // meaning that it is not expedient to get an // updated response from the OCSP server. func freshOCSP(resp *ocsp.Response) bool { nextUpdate := resp.NextUpdate // If there is an OCSP responder certificate, and it expires before the // OCSP response, use its expiration date as the end of the OCSP // response's validity period. if resp.Certificate != nil && resp.Certificate.NotAfter.Before(nextUpdate) { nextUpdate = resp.Certificate.NotAfter } // start checking OCSP staple about halfway through validity period for good measure refreshTime := resp.ThisUpdate.Add(nextUpdate.Sub(resp.ThisUpdate) / 2) return time.Now().Before(refreshTime) } golang-github-caddyserver-certmagic-0.15.2/ratelimiter.go000066400000000000000000000151161414454377300234630ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "log" "runtime" "sync" "time" ) // NewRateLimiter returns a rate limiter that allows up to maxEvents // in a sliding window of size window. If maxEvents and window are // both 0, or if maxEvents is non-zero and window is 0, rate limiting // is disabled. This function panics if maxEvents is less than 0 or // if maxEvents is 0 and window is non-zero, which is considered to be // an invalid configuration, as it would never allow events. func NewRateLimiter(maxEvents int, window time.Duration) *RingBufferRateLimiter { if maxEvents < 0 { panic("maxEvents cannot be less than zero") } if maxEvents == 0 && window != 0 { panic("invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } rbrl := &RingBufferRateLimiter{ window: window, ring: make([]time.Time, maxEvents), started: make(chan struct{}), stopped: make(chan struct{}), ticket: make(chan struct{}), } go rbrl.loop() <-rbrl.started // make sure loop is ready to receive before we return return rbrl } // RingBufferRateLimiter uses a ring to enforce rate limits // consisting of a maximum number of events within a single // sliding window of a given duration. An empty value is // not valid; use NewRateLimiter to get one. type RingBufferRateLimiter struct { window time.Duration ring []time.Time // maxEvents == len(ring) cursor int // always points to the oldest timestamp mu sync.Mutex // protects ring, cursor, and window started chan struct{} stopped chan struct{} ticket chan struct{} } // Stop cleans up r's scheduling goroutine. func (r *RingBufferRateLimiter) Stop() { close(r.stopped) } func (r *RingBufferRateLimiter) loop() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: ring buffer rate limiter: %v\n%s", err, buf) } }() for { // if we've been stopped, return select { case <-r.stopped: return default: } if len(r.ring) == 0 { if r.window == 0 { // rate limiting is disabled; always allow immediately r.permit() continue } panic("invalid configuration: maxEvents = 0 and window != 0 does not allow any events") } // wait until next slot is available or until we've been stopped r.mu.Lock() then := r.ring[r.cursor].Add(r.window) r.mu.Unlock() waitDuration := time.Until(then) waitTimer := time.NewTimer(waitDuration) select { case <-waitTimer.C: r.permit() case <-r.stopped: waitTimer.Stop() return } } } // Allow returns true if the event is allowed to // happen right now. It does not wait. If the event // is allowed, a ticket is claimed. func (r *RingBufferRateLimiter) Allow() bool { select { case <-r.ticket: return true default: return false } } // Wait blocks until the event is allowed to occur. It returns an // error if the context is cancelled. func (r *RingBufferRateLimiter) Wait(ctx context.Context) error { select { case <-ctx.Done(): return context.Canceled case <-r.ticket: return nil } } // MaxEvents returns the maximum number of events that // are allowed within the sliding window. func (r *RingBufferRateLimiter) MaxEvents() int { r.mu.Lock() defer r.mu.Unlock() return len(r.ring) } // SetMaxEvents changes the maximum number of events that are // allowed in the sliding window. If the new limit is lower, // the oldest events will be forgotten. If the new limit is // higher, the window will suddenly have capacity for new // reservations. It panics if maxEvents is 0 and window size // is not zero. func (r *RingBufferRateLimiter) SetMaxEvents(maxEvents int) { newRing := make([]time.Time, maxEvents) r.mu.Lock() defer r.mu.Unlock() if r.window != 0 && maxEvents == 0 { panic("invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } // only make the change if the new limit is different if maxEvents == len(r.ring) { return } // the new ring may be smaller; fast-forward to the // oldest timestamp that will be kept in the new // ring so the oldest ones are forgotten and the // newest ones will be remembered sizeDiff := len(r.ring) - maxEvents for i := 0; i < sizeDiff; i++ { r.advance() } if len(r.ring) > 0 { // copy timestamps into the new ring until we // have either copied all of them or have reached // the capacity of the new ring startCursor := r.cursor for i := 0; i < len(newRing); i++ { newRing[i] = r.ring[r.cursor] r.advance() if r.cursor == startCursor { // new ring is larger than old one; // "we've come full circle" break } } } r.ring = newRing r.cursor = 0 } // Window returns the size of the sliding window. func (r *RingBufferRateLimiter) Window() time.Duration { r.mu.Lock() defer r.mu.Unlock() return r.window } // SetWindow changes r's sliding window duration to window. // Goroutines that are already blocked on a call to Wait() // will not be affected. It panics if window is non-zero // but the max event limit is 0. func (r *RingBufferRateLimiter) SetWindow(window time.Duration) { r.mu.Lock() defer r.mu.Unlock() if window != 0 && len(r.ring) == 0 { panic("invalid configuration: maxEvents = 0 and window != 0 would not allow any events") } r.window = window } // permit allows one event through the throttle. This method // blocks until a goroutine is waiting for a ticket or until // the rate limiter is stopped. func (r *RingBufferRateLimiter) permit() { for { select { case r.started <- struct{}{}: // notify parent goroutine that we've started; should // only happen once, before constructor returns continue case <-r.stopped: return case r.ticket <- struct{}{}: r.mu.Lock() defer r.mu.Unlock() if len(r.ring) > 0 { r.ring[r.cursor] = time.Now() r.advance() } return } } } // advance moves the cursor to the next position. // It is NOT safe for concurrent use, so it must // be called inside a lock on r.mu. func (r *RingBufferRateLimiter) advance() { r.cursor++ if r.cursor >= len(r.ring) { r.cursor = 0 } } golang-github-caddyserver-certmagic-0.15.2/solvers.go000066400000000000000000000530551414454377300226430ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "crypto/tls" "encoding/json" "fmt" "log" "net" "net/http" "path" "runtime" "strings" "sync" "sync/atomic" "time" "github.com/libdns/libdns" "github.com/mholt/acmez" "github.com/mholt/acmez/acme" "github.com/miekg/dns" ) // httpSolver solves the HTTP challenge. It must be // associated with a config and an address to use // for solving the challenge. If multiple httpSolvers // are initialized concurrently, the first one to // begin will start the server, and the last one to // finish will stop the server. This solver must be // wrapped by a distributedSolver to work properly, // because the only way the HTTP challenge handler // can access the keyAuth material is by loading it // from storage, which is done by distributedSolver. type httpSolver struct { closed int32 // accessed atomically acmeManager *ACMEManager address string } // Present starts an HTTP server if none is already listening on s.address. func (s *httpSolver) Present(ctx context.Context, _ acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count++ if si.listener != nil { return nil // already be served by us } // notice the unusual error handling here; we // only continue to start a challenge server if // we got a listener; in all other cases return ln, err := robustTryListen(s.address) if ln == nil { return err } // successfully bound socket, so save listener and start key auth HTTP server si.listener = ln go s.serve(si) return nil } // serve is an HTTP server that serves only HTTP challenge responses. func (s *httpSolver) serve(si *solverInfo) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: http solver server: %v\n%s", err, buf) } }() defer close(si.done) httpServer := &http.Server{Handler: s.acmeManager.HTTPChallengeHandler(http.NewServeMux())} httpServer.SetKeepAlivesEnabled(false) err := httpServer.Serve(si.listener) if err != nil && atomic.LoadInt32(&s.closed) != 1 { log.Printf("[ERROR] key auth HTTP server: %v", err) } } // CleanUp cleans up the HTTP server if it is the last one to finish. func (s *httpSolver) CleanUp(ctx context.Context, _ acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count-- if si.count == 0 { // last one out turns off the lights atomic.StoreInt32(&s.closed, 1) if si.listener != nil { si.listener.Close() <-si.done } delete(solvers, s.address) } return nil } // tlsALPNSolver is a type that can solve TLS-ALPN challenges. // It must have an associated config and address on which to // serve the challenge. type tlsALPNSolver struct { config *Config address string } // Present adds the certificate to the certificate cache and, if // needed, starts a TLS server for answering TLS-ALPN challenges. func (s *tlsALPNSolver) Present(ctx context.Context, chal acme.Challenge) error { // we pre-generate the certificate for efficiency with multi-perspective // validation, so it only has to be done once (at least, by this instance; // distributed solving does not have that luxury, oh well) - update the // challenge data in memory to be the generated certificate cert, err := acmez.TLSALPN01ChallengeCert(chal) if err != nil { return err } key := challengeKey(chal) activeChallengesMu.Lock() chalData := activeChallenges[key] chalData.data = cert activeChallenges[key] = chalData activeChallengesMu.Unlock() // the rest of this function increments the // challenge count for the solver at this // listener address, and if necessary, starts // a simple TLS server solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count++ if si.listener != nil { return nil // already be served by us } // notice the unusual error handling here; we // only continue to start a challenge server if // we got a listener; in all other cases return ln, err := robustTryListen(s.address) if ln == nil { return err } // we were able to bind the socket, so make it into a TLS // listener, store it with the solverInfo, and start the // challenge server si.listener = tls.NewListener(ln, s.config.TLSConfig()) go func() { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: tls-alpn solver server: %v\n%s", err, buf) } }() defer close(si.done) for { conn, err := si.listener.Accept() if err != nil { if atomic.LoadInt32(&si.closed) == 1 { return } log.Printf("[ERROR] TLS-ALPN challenge server: accept: %v", err) continue } go s.handleConn(conn) } }() return nil } // handleConn completes the TLS handshake and then closes conn. func (*tlsALPNSolver) handleConn(conn net.Conn) { defer func() { if err := recover(); err != nil { buf := make([]byte, stackTraceBufferSize) buf = buf[:runtime.Stack(buf, false)] log.Printf("panic: tls-alpn solver handler: %v\n%s", err, buf) } }() defer conn.Close() tlsConn, ok := conn.(*tls.Conn) if !ok { log.Printf("[ERROR] TLS-ALPN challenge server: expected tls.Conn but got %T: %#v", conn, conn) return } err := tlsConn.Handshake() if err != nil { log.Printf("[ERROR] TLS-ALPN challenge server: handshake: %v", err) return } } // CleanUp removes the challenge certificate from the cache, and if // it is the last one to finish, stops the TLS server. func (s *tlsALPNSolver) CleanUp(ctx context.Context, chal acme.Challenge) error { solversMu.Lock() defer solversMu.Unlock() si := getSolverInfo(s.address) si.count-- if si.count == 0 { // last one out turns off the lights atomic.StoreInt32(&si.closed, 1) if si.listener != nil { si.listener.Close() <-si.done } delete(solvers, s.address) } return nil } // DNS01Solver is a type that makes libdns providers usable // as ACME dns-01 challenge solvers. // See https://github.com/libdns/libdns type DNS01Solver struct { // The implementation that interacts with the DNS // provider to set or delete records. (REQUIRED) DNSProvider ACMEDNSProvider // The TTL for the temporary challenge records. TTL time.Duration // Maximum time to wait for temporary record to appear. PropagationTimeout time.Duration // Preferred DNS resolver(s) to use when doing DNS lookups. Resolvers []string txtRecords map[string]dnsPresentMemory // keyed by domain name txtRecordsMu sync.Mutex } // Present creates the DNS TXT record for the given ACME challenge. func (s *DNS01Solver) Present(ctx context.Context, challenge acme.Challenge) error { dnsName := challenge.DNS01TXTRecordName() keyAuth := challenge.DNS01KeyAuthorization() // multiple identifiers can have the same ACME challenge // domain (e.g. example.com and *.example.com) so we need // to ensure that we don't solve those concurrently and // step on each challenges' metaphorical toes; see // https://github.com/caddyserver/caddy/issues/3474 activeDNSChallenges.Lock(dnsName) zone, err := findZoneByFQDN(dnsName, recursiveNameservers(s.Resolvers)) if err != nil { return fmt.Errorf("could not determine zone for domain %q: %v", dnsName, err) } rec := libdns.Record{ Type: "TXT", Name: libdns.RelativeName(dnsName+".", zone), Value: keyAuth, TTL: s.TTL, } results, err := s.DNSProvider.AppendRecords(ctx, zone, []libdns.Record{rec}) if err != nil { return fmt.Errorf("adding temporary record for zone %s: %w", zone, err) } if len(results) != 1 { return fmt.Errorf("expected one record, got %d: %v", len(results), results) } // remember the record and zone we got so we can clean up more efficiently s.txtRecordsMu.Lock() if s.txtRecords == nil { s.txtRecords = make(map[string]dnsPresentMemory) } s.txtRecords[dnsName] = dnsPresentMemory{dnsZone: zone, rec: results[0]} s.txtRecordsMu.Unlock() return nil } // Wait blocks until the TXT record created in Present() appears in // authoritative lookups, i.e. until it has propagated, or until // timeout, whichever is first. func (s *DNS01Solver) Wait(ctx context.Context, challenge acme.Challenge) error { dnsName := challenge.DNS01TXTRecordName() keyAuth := challenge.DNS01KeyAuthorization() timeout := s.PropagationTimeout if timeout == 0 { timeout = 2 * time.Minute } const interval = 2 * time.Second resolvers := recursiveNameservers(s.Resolvers) var err error start := time.Now() for time.Since(start) < timeout { select { case <-time.After(interval): case <-ctx.Done(): return ctx.Err() } var ready bool ready, err = checkDNSPropagation(dnsName, keyAuth, resolvers) if err != nil { return fmt.Errorf("checking DNS propagation of %s: %w", dnsName, err) } if ready { return nil } } return fmt.Errorf("timed out waiting for record to fully propagate; verify DNS provider configuration is correct - last error: %v", err) } // CleanUp deletes the DNS TXT record created in Present(). func (s *DNS01Solver) CleanUp(ctx context.Context, challenge acme.Challenge) error { dnsName := challenge.DNS01TXTRecordName() defer func() { // always forget about it so we don't leak memory s.txtRecordsMu.Lock() delete(s.txtRecords, dnsName) s.txtRecordsMu.Unlock() // always do this last - but always do it! activeDNSChallenges.Unlock(dnsName) }() // recall the record we created and zone we looked up s.txtRecordsMu.Lock() memory, ok := s.txtRecords[dnsName] if !ok { s.txtRecordsMu.Unlock() return fmt.Errorf("no memory of presenting a DNS record for %s (probably OK if presenting failed)", challenge.Identifier.Value) } s.txtRecordsMu.Unlock() // clean up the record _, err := s.DNSProvider.DeleteRecords(ctx, memory.dnsZone, []libdns.Record{memory.rec}) if err != nil { return fmt.Errorf("deleting temporary record for zone %s: %w", memory.dnsZone, err) } return nil } type dnsPresentMemory struct { dnsZone string rec libdns.Record } // ACMEDNSProvider defines the set of operations required for // ACME challenges. A DNS provider must be able to append and // delete records in order to solve ACME challenges. Find one // you can use at https://github.com/libdns. If your provider // isn't implemented yet, feel free to contribute! type ACMEDNSProvider interface { libdns.RecordAppender libdns.RecordDeleter } // activeDNSChallenges synchronizes DNS challenges for // names to ensure that challenges for the same ACME // DNS name do not overlap; for example, the TXT record // to make for both example.com and *.example.com are // the same; thus we cannot solve them concurrently. var activeDNSChallenges = newMapMutex() // mapMutex implements named mutexes. type mapMutex struct { cond *sync.Cond set map[interface{}]struct{} } func newMapMutex() *mapMutex { return &mapMutex{ cond: sync.NewCond(new(sync.Mutex)), set: make(map[interface{}]struct{}), } } func (mmu *mapMutex) Lock(key interface{}) { mmu.cond.L.Lock() defer mmu.cond.L.Unlock() for mmu.locked(key) { mmu.cond.Wait() } mmu.set[key] = struct{}{} } func (mmu *mapMutex) Unlock(key interface{}) { mmu.cond.L.Lock() defer mmu.cond.L.Unlock() delete(mmu.set, key) mmu.cond.Broadcast() } func (mmu *mapMutex) locked(key interface{}) (ok bool) { _, ok = mmu.set[key] return } // distributedSolver allows the ACME HTTP-01 and TLS-ALPN challenges // to be solved by an instance other than the one which initiated it. // This is useful behind load balancers or in other cluster/fleet // configurations. The only requirement is that the instance which // initiates the challenge shares the same storage and locker with // the others in the cluster. The storage backing the certificate // cache in distributedSolver.config is crucial. // // Obviously, the instance which completes the challenge must be // serving on the HTTPChallengePort for the HTTP-01 challenge or the // TLSALPNChallengePort for the TLS-ALPN-01 challenge (or have all // the packets port-forwarded) to receive and handle the request. The // server which receives the challenge must handle it by checking to // see if the challenge token exists in storage, and if so, decode it // and use it to serve up the correct response. HTTPChallengeHandler // in this package as well as the GetCertificate method implemented // by a Config support and even require this behavior. // // In short: the only two requirements for cluster operation are // sharing sync and storage, and using the facilities provided by // this package for solving the challenges. type distributedSolver struct { // The storage backing the distributed solver. It must be // the same storage configuration as what is solving the // challenge in order to be effective. storage Storage // The storage key prefix, associated with the issuer // that is solving the challenge. storageKeyIssuerPrefix string // Since the distributedSolver is only a // wrapper over an actual solver, place // the actual solver here. solver acmez.Solver } // Present invokes the underlying solver's Present method // and also stores domain, token, and keyAuth to the storage // backing the certificate cache of dhs.acmeManager. func (dhs distributedSolver) Present(ctx context.Context, chal acme.Challenge) error { infoBytes, err := json.Marshal(chal) if err != nil { return err } err = dhs.storage.Store(dhs.challengeTokensKey(challengeKey(chal)), infoBytes) if err != nil { return err } err = dhs.solver.Present(ctx, chal) if err != nil { return fmt.Errorf("presenting with embedded solver: %v", err) } return nil } // Wait wraps the underlying solver's Wait() method, if any. Implements acmez.Waiter. func (dhs distributedSolver) Wait(ctx context.Context, challenge acme.Challenge) error { if waiter, ok := dhs.solver.(acmez.Waiter); ok { return waiter.Wait(ctx, challenge) } return nil } // CleanUp invokes the underlying solver's CleanUp method // and also cleans up any assets saved to storage. func (dhs distributedSolver) CleanUp(ctx context.Context, chal acme.Challenge) error { err := dhs.storage.Delete(dhs.challengeTokensKey(challengeKey(chal))) if err != nil { return err } err = dhs.solver.CleanUp(ctx, chal) if err != nil { return fmt.Errorf("cleaning up embedded provider: %v", err) } return nil } // challengeTokensPrefix returns the key prefix for challenge info. func (dhs distributedSolver) challengeTokensPrefix() string { return path.Join(dhs.storageKeyIssuerPrefix, "challenge_tokens") } // challengeTokensKey returns the key to use to store and access // challenge info for domain. func (dhs distributedSolver) challengeTokensKey(domain string) string { return path.Join(dhs.challengeTokensPrefix(), StorageKeys.Safe(domain)+".json") } // solverInfo associates a listener with the // number of challenges currently using it. type solverInfo struct { closed int32 // accessed atomically count int listener net.Listener done chan struct{} // used to signal when our own solver server is done } // getSolverInfo gets a valid solverInfo struct for address. func getSolverInfo(address string) *solverInfo { si, ok := solvers[address] if !ok { si = &solverInfo{done: make(chan struct{})} solvers[address] = si } return si } // robustTryListen calls net.Listen for a TCP socket at addr. // This function may return both a nil listener and a nil error! // If it was able to bind the socket, it returns the listener // and no error. If it wasn't able to bind the socket because // the socket is already in use, then it returns a nil listener // and nil error. If it had any other error, it returns the // error. The intended error handling logic for this function // is to proceed if the returned listener is not nil; otherwise // return err (which may also be nil). In other words, this // function ignores errors if the socket is already in use, // which is useful for our challenge servers, where we assume // that whatever is already listening can solve the challenges. func robustTryListen(addr string) (net.Listener, error) { var listenErr error for i := 0; i < 2; i++ { // doesn't hurt to sleep briefly before the second // attempt in case the OS has timing issues if i > 0 { time.Sleep(100 * time.Millisecond) } // if we can bind the socket right away, great! var ln net.Listener ln, listenErr = net.Listen("tcp", addr) if listenErr == nil { return ln, nil } // if it failed just because the socket is already in use, we // have no choice but to assume that whatever is using the socket // can answer the challenge already, so we ignore the error connectErr := dialTCPSocket(addr) if connectErr == nil { return nil, nil } // hmm, we couldn't connect to the socket, so something else must // be wrong, right? wrong!! we've had reports across multiple OSes // now that sometimes connections fail even though the OS told us // that the address was already in use; either the listener is // fluctuating between open and closed very, very quickly, or the // OS is inconsistent and contradicting itself; I have been unable // to reproduce this, so I'm now resorting to hard-coding substring // matching in error messages as a really hacky and unreliable // safeguard against this, until we can idenify exactly what was // happening; see the following threads for more info: // https://caddy.community/t/caddy-retry-error/7317 // https://caddy.community/t/v2-upgrade-to-caddy2-failing-with-errors/7423 if strings.Contains(listenErr.Error(), "address already in use") || strings.Contains(listenErr.Error(), "one usage of each socket address") { log.Printf("[WARNING] OS reports a contradiction: %v - but we cannot connect to it, with this error: %v; continuing anyway 🤞 (I don't know what causes this... if you do, please help?)", listenErr, connectErr) return nil, nil } } return nil, fmt.Errorf("could not start listener for challenge server at %s: %v", addr, listenErr) } // dialTCPSocket connects to a TCP address just for the sake of // seeing if it is open. It returns a nil error if a TCP connection // can successfully be made to addr within a short timeout. func dialTCPSocket(addr string) error { conn, err := net.DialTimeout("tcp", addr, 250*time.Millisecond) if err == nil { conn.Close() } return err } // GetACMEChallenge returns an active ACME challenge for the given identifier, // or false if no active challenge for that identifier is known. func GetACMEChallenge(identifier string) (Challenge, bool) { activeChallengesMu.Lock() chalData, ok := activeChallenges[identifier] activeChallengesMu.Unlock() return chalData, ok } // The active challenge solvers, keyed by listener address, // and protected by a mutex. Note that the creation of // solver listeners and the incrementing of their counts // are atomic operations guarded by this mutex. var ( solvers = make(map[string]*solverInfo) solversMu sync.Mutex ) // activeChallenges holds information about all known, currently-active // ACME challenges, keyed by identifier. CertMagic guarantees that // challenges for the same identifier do not overlap, by its locking // mechanisms; thus if a challenge comes in for a certain identifier, // we can be confident that if this process initiated the challenge, // the correct information to solve it is in this map. (It may have // alternatively been initiated by another instance in a cluster, in // which case the distributed solver will take care of that.) var ( activeChallenges = make(map[string]Challenge) activeChallengesMu sync.Mutex ) // Challenge is an ACME challenge, but optionally paired with // data that can make it easier or more efficient to solve. type Challenge struct { acme.Challenge data interface{} } // challengeKey returns the map key for a given challenge; it is the identifier // unless it is an IP address using the TLS-ALPN challenge. func challengeKey(chal acme.Challenge) string { if chal.Type == acme.ChallengeTypeTLSALPN01 && chal.Identifier.Type == "ip" { reversed, err := dns.ReverseAddr(chal.Identifier.Value) if err == nil { return reversed[:len(reversed)-1] // strip off '.' } } return chal.Identifier.Value } // solverWrapper should be used to wrap all challenge solvers so that // we can add the challenge info to memory; this makes challenges globally // solvable by a single HTTP or TLS server even if multiple servers with // different configurations/scopes need to get certificates. type solverWrapper struct{ acmez.Solver } func (sw solverWrapper) Present(ctx context.Context, chal acme.Challenge) error { activeChallengesMu.Lock() activeChallenges[challengeKey(chal)] = Challenge{Challenge: chal} activeChallengesMu.Unlock() return sw.Solver.Present(ctx, chal) } func (sw solverWrapper) Wait(ctx context.Context, chal acme.Challenge) error { if waiter, ok := sw.Solver.(acmez.Waiter); ok { return waiter.Wait(ctx, chal) } return nil } func (sw solverWrapper) CleanUp(ctx context.Context, chal acme.Challenge) error { activeChallengesMu.Lock() delete(activeChallenges, challengeKey(chal)) activeChallengesMu.Unlock() return sw.Solver.CleanUp(ctx, chal) } // Interface guards var ( _ acmez.Solver = (*solverWrapper)(nil) _ acmez.Waiter = (*solverWrapper)(nil) _ acmez.Waiter = (*distributedSolver)(nil) ) golang-github-caddyserver-certmagic-0.15.2/solvers_test.go000066400000000000000000000064421414454377300237000ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "testing" "github.com/mholt/acmez/acme" ) func Test_challengeKey(t *testing.T) { type args struct { chal acme.Challenge } tests := []struct { name string args args want string }{ { name: "ok/dns-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeDNS01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/http-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/tls-dns", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "dns", Value: "*.example.com", }, }, }, want: "*.example.com", }, { name: "ok/http-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1", }, }, }, want: "127.0.0.1", }, { name: "ok/http-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeHTTP01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::1", }, }, }, want: "2001:db8::1", }, { name: "ok/tls-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1", }, }, }, want: "1.0.0.127.in-addr.arpa", }, { name: "ok/tls-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::1", }, }, }, want: "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.b.d.0.1.0.0.2.ip6.arpa", }, { name: "fail/tls-ipv4", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "127.0.0.1000", }, }, }, want: "127.0.0.1000", // reversing this fails; default to identifier value }, { name: "fail/tls-ipv6", args: args{ chal: acme.Challenge{ Type: acme.ChallengeTypeTLSALPN01, Identifier: acme.Identifier{ Type: "ip", Value: "2001:db8::10000", }, }, }, want: "2001:db8::10000", // reversing this fails; default to identifier value }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { if got := challengeKey(tt.args.chal); got != tt.want { t.Errorf("challengeKey() = %v, want %v", got, tt.want) } }) } } golang-github-caddyserver-certmagic-0.15.2/storage.go000066400000000000000000000217741414454377300226150ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "context" "path" "regexp" "strings" "sync" "time" "go.uber.org/zap" ) // Storage is a type that implements a key-value store. // Keys are prefix-based, with forward slash '/' as separators // and without a leading slash. // // Processes running in a cluster will wish to use the // same Storage value (its implementation and configuration) // in order to share certificates and other TLS resources // with the cluster. // // The Load, Delete, List, and Stat methods should return // ErrNotExist if the key does not exist. // // Implementations of Storage must be safe for concurrent use. type Storage interface { // Locker provides atomic synchronization // operations, making Storage safe to share. Locker // Store puts value at key. Store(key string, value []byte) error // Load retrieves the value at key. Load(key string) ([]byte, error) // Delete deletes key. An error should be // returned only if the key still exists // when the method returns. Delete(key string) error // Exists returns true if the key exists // and there was no error checking. Exists(key string) bool // List returns all keys that match prefix. // If recursive is true, non-terminal keys // will be enumerated (i.e. "directories" // should be walked); otherwise, only keys // prefixed exactly by prefix will be listed. List(prefix string, recursive bool) ([]string, error) // Stat returns information about key. Stat(key string) (KeyInfo, error) } // Locker facilitates synchronization of certificate tasks across // machines and networks. type Locker interface { // Lock acquires the lock for key, blocking until the lock // can be obtained or an error is returned. Note that, even // after acquiring a lock, an idempotent operation may have // already been performed by another process that acquired // the lock before - so always check to make sure idempotent // operations still need to be performed after acquiring the // lock. // // The actual implementation of obtaining of a lock must be // an atomic operation so that multiple Lock calls at the // same time always results in only one caller receiving the // lock at any given time. // // To prevent deadlocks, all implementations (where this concern // is relevant) should put a reasonable expiration on the lock in // case Unlock is unable to be called due to some sort of network // failure or system crash. Additionally, implementations should // honor context cancellation as much as possible (in case the // caller wishes to give up and free resources before the lock // can be obtained). Lock(ctx context.Context, key string) error // Unlock releases the lock for key. This method must ONLY be // called after a successful call to Lock, and only after the // critical section is finished, even if it errored or timed // out. Unlock cleans up any resources allocated during Lock. Unlock(key string) error } // KeyInfo holds information about a key in storage. // Key and IsTerminal are required; Modified and Size // are optional if the storage implementation is not // able to get that information. Setting them will // make certain operations more consistent or // predictable, but it is not crucial to basic // functionality. type KeyInfo struct { Key string Modified time.Time Size int64 IsTerminal bool // false for keys that only contain other keys (like directories) } // storeTx stores all the values or none at all. func storeTx(s Storage, all []keyValue) error { for i, kv := range all { err := s.Store(kv.key, kv.value) if err != nil { for j := i - 1; j >= 0; j-- { s.Delete(all[j].key) } return err } } return nil } // keyValue pairs a key and a value. type keyValue struct { key string value []byte } // KeyBuilder provides a namespace for methods that // build keys and key prefixes, for addressing items // in a Storage implementation. type KeyBuilder struct{} // CertsPrefix returns the storage key prefix for // the given certificate issuer. func (keys KeyBuilder) CertsPrefix(issuerKey string) string { return path.Join(prefixCerts, keys.Safe(issuerKey)) } // CertsSitePrefix returns a key prefix for items associated with // the site given by domain using the given issuer key. func (keys KeyBuilder) CertsSitePrefix(issuerKey, domain string) string { return path.Join(keys.CertsPrefix(issuerKey), keys.Safe(domain)) } // SiteCert returns the path to the certificate file for domain // that is associated with the issuer with the given issuerKey. func (keys KeyBuilder) SiteCert(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".crt") } // SitePrivateKey returns the path to the private key file for domain // that is associated with the certificate from the given issuer with // the given issuerKey. func (keys KeyBuilder) SitePrivateKey(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".key") } // SiteMeta returns the path to the metadata file for domain that // is associated with the certificate from the given issuer with // the given issuerKey. func (keys KeyBuilder) SiteMeta(issuerKey, domain string) string { safeDomain := keys.Safe(domain) return path.Join(keys.CertsSitePrefix(issuerKey, domain), safeDomain+".json") } // OCSPStaple returns a key for the OCSP staple associated // with the given certificate. If you have the PEM bundle // handy, pass that in to save an extra encoding step. func (keys KeyBuilder) OCSPStaple(cert *Certificate, pemBundle []byte) string { var ocspFileName string if len(cert.Names) > 0 { firstName := keys.Safe(cert.Names[0]) ocspFileName = firstName + "-" } ocspFileName += fastHash(pemBundle) return path.Join(prefixOCSP, ocspFileName) } // Safe standardizes and sanitizes str for use as // a single component of a storage key. This method // is idempotent. func (keys KeyBuilder) Safe(str string) string { str = strings.ToLower(str) str = strings.TrimSpace(str) // replace a few specific characters repl := strings.NewReplacer( " ", "_", "+", "_plus_", "*", "wildcard_", ":", "-", "..", "", // prevent directory traversal (regex allows single dots) ) str = repl.Replace(str) // finally remove all non-word characters return safeKeyRE.ReplaceAllLiteralString(str, "") } // CleanUpOwnLocks immediately cleans up all // current locks obtained by this process. Since // this does not cancel the operations that // the locks are synchronizing, this should be // called only immediately before process exit. // Errors are only reported if a logger is given. func CleanUpOwnLocks(logger *zap.Logger) { locksMu.Lock() defer locksMu.Unlock() for lockKey, storage := range locks { err := storage.Unlock(lockKey) if err == nil { delete(locks, lockKey) } else if logger != nil { logger.Error("unable to clean up lock in storage backend", zap.Any("storage", storage), zap.String("lock_key", lockKey), zap.Error(err), ) } } } func acquireLock(ctx context.Context, storage Storage, lockKey string) error { err := storage.Lock(ctx, lockKey) if err == nil { locksMu.Lock() locks[lockKey] = storage locksMu.Unlock() } return err } func releaseLock(storage Storage, lockKey string) error { err := storage.Unlock(lockKey) if err == nil { locksMu.Lock() delete(locks, lockKey) locksMu.Unlock() } return err } // locks stores a reference to all the current // locks obtained by this process. var locks = make(map[string]Storage) var locksMu sync.Mutex // StorageKeys provides methods for accessing // keys and key prefixes for items in a Storage. // Typically, you will not need to use this // because accessing storage is abstracted away // for most cases. Only use this if you need to // directly access TLS assets in your application. var StorageKeys KeyBuilder const ( prefixCerts = "certificates" prefixOCSP = "ocsp" ) // safeKeyRE matches any undesirable characters in storage keys. // Note that this allows dots, so you'll have to strip ".." manually. var safeKeyRE = regexp.MustCompile(`[^\w@.-]`) // ErrNotExist is returned by Storage implementations when // a resource is not found. It is similar to os.IsNotExist // except this is a type, not a variable. // TODO: use new Go error wrapping conventions type ErrNotExist interface { error } // defaultFileStorage is a convenient, default storage // implementation using the local file system. var defaultFileStorage = &FileStorage{Path: dataDir()} golang-github-caddyserver-certmagic-0.15.2/storage_test.go000066400000000000000000000055441414454377300236510ustar00rootroot00000000000000// Copyright 2015 Matthew Holt // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // See the License for the specific language governing permissions and // limitations under the License. package certmagic import ( "path" "testing" ) func TestPrefixAndKeyBuilders(t *testing.T) { am := &ACMEManager{CA: "https://example.com/acme-ca/directory"} base := path.Join("certificates", "example.com-acme-ca-directory") for i, testcase := range []struct { in, folder, certFile, keyFile, metaFile string }{ { in: "example.com", folder: path.Join(base, "example.com"), certFile: path.Join(base, "example.com", "example.com.crt"), keyFile: path.Join(base, "example.com", "example.com.key"), metaFile: path.Join(base, "example.com", "example.com.json"), }, { in: "*.example.com", folder: path.Join(base, "wildcard_.example.com"), certFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.crt"), keyFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.key"), metaFile: path.Join(base, "wildcard_.example.com", "wildcard_.example.com.json"), }, { // prevent directory traversal! very important, esp. with on-demand TLS // see issue #2092 in: "a/../../../foo", folder: path.Join(base, "afoo"), certFile: path.Join(base, "afoo", "afoo.crt"), keyFile: path.Join(base, "afoo", "afoo.key"), metaFile: path.Join(base, "afoo", "afoo.json"), }, { in: "b\\..\\..\\..\\foo", folder: path.Join(base, "bfoo"), certFile: path.Join(base, "bfoo", "bfoo.crt"), keyFile: path.Join(base, "bfoo", "bfoo.key"), metaFile: path.Join(base, "bfoo", "bfoo.json"), }, { in: "c/foo", folder: path.Join(base, "cfoo"), certFile: path.Join(base, "cfoo", "cfoo.crt"), keyFile: path.Join(base, "cfoo", "cfoo.key"), metaFile: path.Join(base, "cfoo", "cfoo.json"), }, } { if actual := StorageKeys.SiteCert(am.IssuerKey(), testcase.in); actual != testcase.certFile { t.Errorf("Test %d: site cert file: Expected '%s' but got '%s'", i, testcase.certFile, actual) } if actual := StorageKeys.SitePrivateKey(am.IssuerKey(), testcase.in); actual != testcase.keyFile { t.Errorf("Test %d: site key file: Expected '%s' but got '%s'", i, testcase.keyFile, actual) } if actual := StorageKeys.SiteMeta(am.IssuerKey(), testcase.in); actual != testcase.metaFile { t.Errorf("Test %d: site meta file: Expected '%s' but got '%s'", i, testcase.metaFile, actual) } } } golang-github-caddyserver-certmagic-0.15.2/testdata/000077500000000000000000000000001414454377300224205ustar00rootroot00000000000000golang-github-caddyserver-certmagic-0.15.2/testdata/resolv.conf.1000066400000000000000000000002031414454377300247330ustar00rootroot00000000000000domain company.com nameserver 10.200.3.249 nameserver 10.200.3.250:5353 nameserver 2001:4860:4860::8844 nameserver [10.0.0.1]:5353