pax_global_header00006660000000000000000000000064131202662040014506gustar00rootroot0000000000000052 comment=42d97b9475e23f854f5f785aa05148d8e6658751 clj-http-client-0.9.0/000077500000000000000000000000001312026620400145155ustar00rootroot00000000000000clj-http-client-0.9.0/.gitignore000066400000000000000000000001601312026620400165020ustar00rootroot00000000000000target pom.xml .nrepl-port /.lein* /resources/locales.clj /resources/**/Messages*.class /dev-resources/i18n/bin clj-http-client-0.9.0/.travis.yml000066400000000000000000000010571312026620400166310ustar00rootroot00000000000000language: clojure lein: 2.7.1 jdk: - oraclejdk7 - openjdk7 script: ./ext/travisci/test.sh notifications: email: false # Java needs to be able to resolve its hostname when doing integration style # tests, which it cannot do in certain cases with travis-ci. If we need the # runtime/container to be able to resolve its own hostname we need to use # either the `hostname` or `hosts` "addon" for travis. Since we don't care # what the hostname is, here we just give it a garbage name based on the name # of the project. addons: hostname: cljhttpclient clj-http-client-0.9.0/CHANGELOG.md000066400000000000000000000153711312026620400163350ustar00rootroot00000000000000## 0.9.0 This is a feature release * [TK-443](https://tickets.puppetlabs.com/browse/TK-443) Adds `:enable-url-metrics?` and `setEnableURLMetrics` to the clojure and java APIs respectively. This allows disabling autogenerated url based metrics. * [PE-19979](https://tickets.puppetlabs.com/browse/PE-19979) & [SERVER-1733](https://tickets.puppetlabs.com/browse/SERVER-1733) Update dependencies via clj-parent (i18n dependency specifically). ## 0.8.0 This is a feature release * [PDB-2640](https://tickets.puppetlabs.com/browse/PDB-2640) Added a `:compress-request-body` request option which allows for the request body content to be gzip-compressed before forwarding it on to the server ## 0.7.0 This is a feature release * [SERVER-1556](https://tickets.puppetlabs.com/browse/SERVER-1556) Add puppetlabs/clj-i18n support to clj-http-client. * Pass the global i18n locale binding in the "accept-language" header of Clojure client requests. ## 0.6.0 This is a feature and maintenance release * [TK-179](https://tickets.puppetlabs.com/browse/TK-179) Refactor to eliminate Clojure/Java duplication * [TK-316](https://tickets.puppetlabs.com/browse/TK-316) Add support for metrics * [TK-317](https://tickets.puppetlabs.com/browse/TK-317) Add ability to pass a metrics-id * [TK-354](https://tickets.puppetlabs.com/browse/TK-354) Add [documentation on metrics](./doc/metrics.md) * [TK-308](https://tickets.puppetlabs.com/browse/TK-308) Bump Apache HttpAsyncClient library to 4.1.2 * [TK-402](https://tickets.puppetlabs.com/browse/TK-402) Allow metric namespace to be configurable * Bump Trapperkeeper and Kitchensink dependencies from 1.1.1 to 1.5.1 (TK) and 1.2.0 to 1.3.0 (KS). ## 0.5.0 This is a feature release. * [TK-312](https://tickets.puppetlabs.com/browse/TK-312) Unbuffered streams in Java to match clojure support released in 0.4.5. * Add request function to clojure client protocol. ## 0.4.6 This is a maintenance release. * [TK-303](https://tickets.puppetlabs.com/browse/TK-303) - update dependencies to use Apache httpasyncclient v4.1.1. ## 0.4.5 This is a feature release, and probably should have been 0.5.0 in order to comply with semantic versioning. * Add support for streaming responses in the Clojure API - (ca5ad63) Scott Walker ## 0.4.4 This is a maintenance release. * [TK-196](https://tickets.puppetlabs.com/browse/TK-196) - update prismatic dependencies to the latest versions. ## 0.4.3 This is a feature release. * [TK-134](https://tickets.puppetlabs.com/browse/TK-134) - Introduced two new optional client configuration settings: - `connect-timeout-milliseconds`: maximum number of milliseconds that the client will wait for a connection to be established. - `socket-timeout-milliseconds`: maximum number of milliseconds that the client will allow for no data to be available on the socket before closing the underlying connection, 'SO_TIMEOUT' in socket terms. ## 0.4.2 This is a bugfix release. * [TK-145](https://tickets.puppetlabs.com/browse/TK-145) - Fixed a bug which caused some HTTP requests to incorrectly have `charset=UTF-8` added to their `Content-Type` headers. ## 0.4.1 This is a maintenance release. * Add documentation for making requests using the Java and Clojure clients. * Upgrade jvm-ssl-utils (previously known as jvm-certificate-authority) dependency to 0.7.0. * Upgrade clj-kitchensink dependency to 1.0.0. * Upgrade trapperkeeper dependency to 1.0.1. ## 0.4.0 This is a feature release which has some breaking changes. * Support for non-client bound asynchronous requests has been removed from both the Clojure and Java-layer APIs. This includes all of the request functions that previously existed in the `client.async` Clojure namespace and the request methods in the `AsyncHttpClient` Java class. * Add a Java-layer API for getting an instance of an HttpClient on which multiple requests -- e.g.., GET, POST -- can be made. Clients are created via the `createClient` method on the new `Async` and `Sync` classes, for a client that can make asynchronous or synchronous web requests, respectively. * Non-client bound synchronous requests can still be performed through the Java API but must now be done through the `Sync` classes rather than the `SyncHttpClient` class. The `SyncHttpClient` class is now used as the type of the instance that the `Sync.createClient()` method returns. * The Java `RequestOptions` class was refactored into new `ClientOptions` and `RequestOptions` classes which can be used with the client-bound `Async` and `Sync` APIs. For non-client bound requests, options are now defined via a `SimpleRequestOptions` class. * Reworked connection close behavior to more robustly handle successful and failed connections. ## 0.3.1 This is a bugfix release. * Fix a memory leak that occurred as a result of connection failures. ## 0.3.0 This is a feature release. * Add configuration settings for SSL Protocols and Cipher Suites to both the Java and Clojure clients. ## 0.2.8 This is a bugfix release. * Fix a bug in the Java client API that caused a file descriptor leak each time a client was created for a new request. ## 0.2.7 This is a bugfix release. * Fix a bug where the character encoding was always being set to ISO-8859-1 w/o a charset ever being explicitly specified in the Content-Type header. We now honor the existing charset if there is one in the header, and otherwise we use UTF-8 and explicitly add the charset to the header. ## 0.2.6 * Add :follow-redirects and :force-redirects options to the clojure client. * Add followRedirects and forceRedirects options to the Java client. ## 0.2.5 * Add an overloaded constructor for `RequestOptions` that accepts a String uri ## 0.2.4 * Fix a bug in the Java client API that caused an NPE if a Content-Type header did not specify a charset ## 0.2.3 * No changes ## 0.2.2 * Add back in support for query-params map in Clojure API ## 0.2.1 * Upgrade to Apache HttpAsyncClient v4.0.2 (fixes a bug where headers don't get included when following redirects). ## 0.2.0 * Port the code to use the Apache HttpAsyncClient library instead of http-kit. * The API around creating a persistent client has changed and persistent clients are explicitly managed * The available request options have changed. Some convenience options have been removed. ## 0.1.7 * Explicitly target JDK6 when building release jars ## 0.1.6 * Add support for configuring client SSL context with a CA cert without a client cert/key ## 0.1.5 * Update to latest version of puppetlabs/kitchensink * Use puppetlabs/certificate-authority for SSL tasks ## 0.1.4 * Fix bug in sync.clj when excluding clojure.core/get ## 0.1.3 * Added a Java API for the synchronous client clj-http-client-0.9.0/CONTRIBUTING.md000066400000000000000000000010171312026620400167450ustar00rootroot00000000000000# How to contribute Third-party patches are essential for keeping puppet open-source projects great. We want to keep it as easy as possible to contribute changes that allow you to get the most out of our projects. There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things. For more info, see our canonical guide to contributing: [https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md](https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md) clj-http-client-0.9.0/LICENSE000066400000000000000000000260751312026620400155340ustar00rootroot00000000000000Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright {yyyy} {name of copyright owner} Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. clj-http-client-0.9.0/MAINTAINERS000066400000000000000000000011651312026620400162150ustar00rootroot00000000000000{ "version": 1, "file_format": "This MAINTAINERS file format is described at http://pup.pt/maintainers", "issues": "https://tickets.puppetlabs.com/browse/TK", "internal_list": "https://groups.google.com/a/puppet.com/forum/?hl=en#!forum/discuss-trapperkeeper-maintainers", "people": [ { "github": "rlinehan", "email": "ruth@puppet.com", "name": "Ruth Linehan" }, { "github": "scottyw", "email": "scott.walker@puppet.com", "name": "Scott Walker" }, { "github": "camlow325", "email": "jeremy.barlow@puppet.com", "name": "Jeremy Barlow" } ] } clj-http-client-0.9.0/Makefile000066400000000000000000000000441312026620400161530ustar00rootroot00000000000000include dev-resources/Makefile.i18n clj-http-client-0.9.0/README.md000066400000000000000000000022611312026620400157750ustar00rootroot00000000000000# puppetlabs/http-client [![Build Status](https://travis-ci.org/puppetlabs/clj-http-client.png?branch=master)](https://travis-ci.org/puppetlabs/clj-http-client) This is a wrapper around the [Apache HttpAsyncClient library](http://hc.apache.org/httpcomponents-asyncclient-4.0.x/) providing some extra functionality for configuring SSL in a way compatible with Puppet. ## Installation Add the following dependency to your `project.clj` file: [![Clojars Project](http://clojars.org/puppetlabs/http-client/latest-version.svg)](http://clojars.org/puppetlabs/http-client) ## Details Async versions of the http methods are exposed in puppetlabs.http.client.async, and synchronous versions are in puppetlabs.http.client.sync. For information on using these namespaces, see the page on [making requests with clojure clients](doc/clojure-client.md). Additionally, this library allows you to make requests using Java clients. For information on how to do this, see the page on [making requests with java clients](doc/java-client.md). ## Support We use the [Trapperkeeper project on JIRA](https://tickets.puppetlabs.com/browse/TK) for tickets on clj-http-client, although Github issues are welcome too. clj-http-client-0.9.0/dev-resources/000077500000000000000000000000001312026620400173035ustar00rootroot00000000000000clj-http-client-0.9.0/dev-resources/Makefile.i18n000066400000000000000000000141011312026620400215160ustar00rootroot00000000000000# -*- Makefile -*- # This file was generated by the i18n leiningen plugin # Do not edit this file; it will be overwritten the next time you run # lein i18n init # # The name of the package into which the translations bundle will be placed BUNDLE=puppetlabs.http_client # The name of the POT file into which the gettext code strings (msgid) will be placed POT_NAME=http-client.pot # The list of names of packages covered by the translation bundle; # by default it contains a single package - the same where the translations # bundle itself is placed - but this can be overridden - preferably in # the top level Makefile PACKAGES?=$(BUNDLE) LOCALES=$(basename $(notdir $(wildcard locales/*.po))) BUNDLE_DIR=$(subst .,/,$(BUNDLE)) BUNDLE_FILES=$(patsubst %,resources/$(BUNDLE_DIR)/Messages_%.class,$(LOCALES)) FIND_SOURCES=find src -name \*.clj # xgettext before 0.19 does not understand --add-location=file. Even CentOS # 7 ships with an older gettext. We will therefore generate full location # info on those systems, and only file names where xgettext supports it LOC_OPT=$(shell xgettext --add-location=file -f - /dev/null 2>&1 && echo --add-location=file || echo --add-location) LOCALES_CLJ=resources/locales.clj define LOCALES_CLJ_CONTENTS { :locales #{$(patsubst %,"%",$(LOCALES))} :packages [$(patsubst %,"%",$(PACKAGES))] :bundle $(patsubst %,"%",$(BUNDLE).Messages) } endef export LOCALES_CLJ_CONTENTS i18n: msgfmt # Update locales/.pot update-pot: locales/$(POT_NAME) locales/$(POT_NAME): $(shell $(FIND_SOURCES)) | locales @tmp=$$(mktemp $@.tmp.XXXX); \ $(FIND_SOURCES) \ | xgettext --from-code=UTF-8 --language=lisp \ --copyright-holder='Puppet ' \ --package-name="$(BUNDLE)" \ --package-version="$(BUNDLE_VERSION)" \ --msgid-bugs-address="docs@puppet.com" \ -k \ -kmark:1 -ki18n/mark:1 \ -ktrs:1 -ki18n/trs:1 \ -ktru:1 -ki18n/tru:1 \ -ktrun:1,2 -ki18n/trun:1,2 \ -ktrsn:1,2 -ki18n/trsn:1,2 \ $(LOC_OPT) \ --add-comments --sort-by-file \ -o $$tmp -f -; \ sed -i.bak -e 's/charset=CHARSET/charset=UTF-8/' $$tmp; \ sed -i.bak -e 's/POT-Creation-Date: [^\\]*/POT-Creation-Date: /' $$tmp; \ rm -f $$tmp.bak; \ if ! diff -q -I POT-Creation-Date $$tmp $@ >/dev/null 2>&1; then \ mv $$tmp $@; \ else \ rm $$tmp; touch $@; \ fi # Run msgfmt over all .po files to generate Java resource bundles # and create the locales.clj file msgfmt: $(BUNDLE_FILES) $(LOCALES_CLJ) clean-orphaned-bundles # Force rebuild of locales.clj if its contents is not the the desired one. The # shell echo is used to add a trailing newline to match the one from `cat` ifneq ($(shell cat $(LOCALES_CLJ) 2> /dev/null),$(shell echo '$(LOCALES_CLJ_CONTENTS)')) .PHONY: $(LOCALES_CLJ) endif $(LOCALES_CLJ): | resources @echo "Writing $@" @echo "$$LOCALES_CLJ_CONTENTS" > $@ # Remove every resource bundle that wasn't generated from a PO file. # We do this because we used to generate the english bundle directly from the POT. .PHONY: clean-orphaned-bundles clean-orphaned-bundles: @for bundle in resources/$(BUNDLE_DIR)/Messages_*.class; do \ locale=$$(basename "$$bundle" | sed -E -e 's/\$$?1?\.class$$/_class/' | cut -d '_' -f 2;); \ if [ ! -f "locales/$$locale.po" ]; then \ rm "$$bundle"; \ fi \ done resources/$(BUNDLE_DIR)/Messages_%.class: locales/%.po | resources msgfmt --java2 -d resources -r $(BUNDLE).Messages -l $(*F) $< # Use this to initialize translations. Updating the PO files is done # automatically through a CI job that utilizes the scripts in the project's # `bin` file, which themselves come from the `clj-i18n` project. locales/%.po: | locales @if [ ! -f $@ ]; then \ touch $@ && msginit --no-translator -l $(*F) -o $@ -i locales/$(POT_NAME); \ fi resources locales: @mkdir $@ help: $(info $(HELP)) @echo .PHONY: help define HELP This Makefile assists in handling i18n related tasks during development. Files that need to be checked into source control are put into the locales/ directory. They are locales/$(POT_NAME) - the POT file generated by 'make update-pot' locales/$$LANG.po - the translations for $$LANG Only the $$LANG.po files should be edited manually; this is usually done by translators. You can use the following targets: i18n: refresh all the files in locales/ and recompile resources update-pot: extract strings and update locales/$(POT_NAME) locales/LANG.po: create translations for LANG msgfmt: compile the translations into Java classes; this step is needed to make translations available to the Clojure code and produces Java class files in resources/ endef # @todo lutter 2015-04-20: for projects that use libraries with their own # translation, we need to combine all their translations into one big po # file and then run msgfmt over that so that we only have to deal with one # resource bundle clj-http-client-0.9.0/dev-resources/java.security000066400000000000000000000040031312026620400220120ustar00rootroot00000000000000# # This is the "override security properties file" which is used by default # in the lein dev profile. End users may override java security properties in # a similar manner in the production code. # # This file augments and overrides $JAVA_HOME/jre/lib/security/java.security # when the java process is provided the option, # -Djava.security.properties=./dev-resources/java.security # # NOTE: It is possible to make this file authoritative, discarding the values # in $JAVA_HOME/jre/lib/security/java.security by setting the first character # of the path to an '=' sign. # # Algorithm restrictions for Secure Socket Layer/Transport Layer Security # (SSL/TLS) processing # In some environments, certain algorithms or key lengths may be undesirable # when using SSL/TLS. This section describes the mechanism for disabling # algorithms during SSL/TLS security parameters negotiation, including # protocol version negotiation, cipher suites selection, peer authentication # and key exchange mechanisms. # # Disabled algorithms will not be negotiated for SSL/TLS connections, even # if they are enabled explicitly in an application. # # For PKI-based peer authentication and key exchange mechanisms, this list # of disabled algorithms will also be checked during certification path # building and validation, including algorithms used in certificates, as # well as revocation information such as CRLs and signed OCSP Responses. # This is in addition to the jdk.certpath.disabledAlgorithms property above. # # See the specification of "jdk.certpath.disabledAlgorithms" for the # syntax of the disabled algorithm string. # # Note: This property is currently used by Oracle's JSSE implementation. # It is not guaranteed to be examined and used by other implementations. # # Example: # jdk.tls.disabledAlgorithms=MD5, SSLv3, DSA, RSA keySize < 2048 # # TK-143 Disable no algorithms so that unit tests are able to exercise the # behavior of the system when the end user explicitly configures deprecated # algorithms like SSLv3. jdk.tls.disabledAlgorithms=clj-http-client-0.9.0/dev-resources/logback-test.xml000066400000000000000000000006261312026620400224100ustar00rootroot00000000000000 %d %-5p [%t] [%c{2}] %m%n clj-http-client-0.9.0/dev-resources/ssl/000077500000000000000000000000001312026620400201045ustar00rootroot00000000000000clj-http-client-0.9.0/dev-resources/ssl/alternate-ca.pem000066400000000000000000000032401312026620400231460ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIEtzCCAp+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAfMR0wGwYDVQQDDBRQdXBw ZXQgQ0E6IGxvY2FsaG9zdDAeFw0xNDAzMjQyMTA2NDNaFw0xOTAzMjUyMTA2NDNa MB8xHTAbBgNVBAMMFFB1cHBldCBDQTogbG9jYWxob3N0MIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAjFXy0dLKBEm9pG9f7x51sdD6HiQHA0kfGbRzNbSx 6psusOO7W3y5xvetpwnd5w+6uLHeNjysHKf4oq70foP3N34cFkEqPfizlYig1aBN Wjq5tVDUQ/wdFuSvxzDVrTWRX5GUebqyyD8VRHcYON1FgeaZacpY7btmdG3kIWq/ Owb8Q4L7DRFJt3aePZ/QGiaTuxQD+CTVY1Wof/i4gGWJeDY3+WHv9b7TlUIU9Mw0 upaS2HuzlC/Xf0iFpr2CbEpYtXfZHS00uQ8ai0/BLrm1ro+D+Qx+4LWPBuEPPRzL 9hO85g3iJbSxgjEytfWE/yvnWZB5AeZKpDD45VfoK41yu53YWVuiLNoisjBTP33I 6d2ufmpmdBU4q+3wfOAd2IkxPNZVYnKvDCul5PP6PoNtxdSV9tDCU19F4hxTb2E6 /BUJQHlvReWNx/E+YAqnW/OFFeRKML8ot4vccAV1FKoVdVOMhih9D5BiYvxf+3PI CSPvteYE0Kr43TdOAhACTV8I8gZL6J0wOuKvBR+wSZFVf0iwo/If4nfvwI0HbEhc 70pTETP08zYzjMoM10qQGnK9dlMpmm5JTElAuMwz2ehN2fyBfQmqEan0sKR2V/pT uS9s79aGBwjvE0/JNy82G4GW7+5q/wALHnj7VW8rooaQd781nuMaUg/KUW9NwNvj jF8CAwEAATANBgkqhkiG9w0BAQsFAAOCAgEAQznz69k0v0RTvWocGfBH+fHwWMmv h+F38U7M/+y8cxZasYhYIxVusIbEl7slzUpu2Yr3pePj63kPk2ctkSVsDGc7SJ84 jpXfzA/s8ZkSmLZpGEAm+pq3AdBJwXaJs2Mvx+JswQoriIkvaKL4q7f1pjB2uf4M w1M12Xs/1GR4mURMr+moHWQw2E0ajC2k/ukwL2Ki5OcIYTeDGhtLvamGj42jjUU1 lN73W9SyvsdzpJlQQkaZcHSRnM+NCVga2H9voVLZVirC8936ebkZrimluozo/BAX JYNvt5MSWhPxGUkNzhzB+2Vwu6rwla6k0NjJCbHVpeXwRCaEDELATeWF0ugsViho 7jT+1o7/tR6q9krCZfCUlNhDN064kjesmn8dNcxl3c6ib4WcUCFx4BE3iuoWPQW/ xI8P/YdHPbi6kXLUYtzOYD0ELOGBXNcdsSNgoRkKpCcSifblHFEKiLoIv7XCgLwZ +1H9NLAsrVfCH+l1DSjbquXJKOQf6n8+0lLIeYLVepfPYEwn2HODuaLNEezl70hu /sMEFJ3cf+xzUyybM8jJRg7GXhE/hojFS6wKaN/uM3m3isHoByn6JI887Ac9MYT1 9kgdqev9ehyGPHpSqQzZj+Js3NfVYY7yKGMqf6WptZ4D1Bt5C7T+0p5JWAsYTZ1O NND5b4a60ptidOk= -----END CERTIFICATE----- clj-http-client-0.9.0/dev-resources/ssl/ca.pem000066400000000000000000000035121312026620400211730ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIFNDCCAxygAwIBAgIBATANBgkqhkiG9w0BAQsFADAfMR0wGwYDVQQDDBRQdXBw ZXQgQ0E6IGV4cGxvc2l2bzAeFw0xMzAxMDEyMTQ0MTZaFw0xODAxMDEyMTQ0MTZa MB8xHTAbBgNVBAMMFFB1cHBldCBDQTogZXhwbG9zaXZvMIICIjANBgkqhkiG9w0B AQEFAAOCAg8AMIICCgKCAgEAv55rxHbHHFuov+Js47hU8n394nGYplAocF88hDgX 6MLG75pzXZ7WqA+uEHxm7KkarEXxBZqkTUlWsxFa0UpgjEQlRDUxnlLsbJL1/HYP SB1+J7a8UNGvZRh2Ich3rbQ8+tv4UQRR8dr7tOQE+EPL0M492FYx/t/0pRHDErKD RJOHxxJLkpapIavObg/bZDEoLf8+ZaU/DyBENJmL4lhHl6i/KhBinuR4UnUfdxZ7 dVTid7gNGk9VGot3x0LWdXYqQ/sxNb/qx6M/0MR5PuO9Pdsmn1SHyRyUcIjX1l6Y H4YH+ryoff0EWEtJxsGrrptXWX7groQpyRZrdknU74cCKyR1JY+7DyV7D8Ixgf1j MU7GKEglVB0qHURWzQgfsNeq3kk9I+dQr8KQvkv83cVYVM7mONA+EyMju8i7u4Oq 5XvpZvBANw+UZwB2lNTDdtBdC+XHQIAM9DYwpLyz20WlAmAZOFzHqe3k6ewwbU+n UEZgauvnmFsN1DbYeLtLabm1P7QgI4pxGfXXhOVCexIDTZ8mv1s3fsCP0tXmKQhp j25FvNJ6k/1xBPKc/PHB2Bms402GaK7lknaYBxIGUzgT2sw/GlcxWiPnV6BUHPwW aivELyu/hRu+Jr6bMsmlC0ZS4ZllrvsAitTPYvHdshQ6BmyHkSr6ZzrcijILGfFg 0esCAwEAAaN7MHkwNwYJYIZIAYb4QgENBCoWKFB1cHBldCBSdWJ5L09wZW5TU0wg SW50ZXJuYWwgQ2VydGlmaWNhdGUwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQF MAMBAf8wHQYDVR0OBBYEFPdrBes9dJY2zNm2UsqhdDwtGMncMA0GCSqGSIb3DQEB CwUAA4ICAQC0oieRsLuDOMYdsUVWj2qsSjjf7SgpdjdBJ23eY9DO1ePz4/MkJG8d 5oWFy9bTBWEDev7/9MWizHtjk/V+AglLaPJB2GyR9zfGeodRx860iqQdhrEs3B9U pGat1kKCDY3yee38Up4I0HyPdjTCPu0FcM+k4ySu1HKvhr0elOo3Y4d/UWGFfc6U bF4K385j1IKtN2O7Iu6DoYZNu+InZ95xiRhApzTRJop3PM+FznYNavFaOlFS5mcj tbKiQjN+8XrpmHJVQniEz4qKY5yZO+IbPBxP8lZOJYvKYtMu2KsbxDd8s/6s40dT SpHu7lEgBe4gQXDMkegHo4npJoiXlHh97pgRgj6DLeUiKtrq31NYRD5gQ+KpKdtg z4mvOcJUXitMV3LtoUGnYNwUXNjvNuMAiFlAQ5fRDTX5dnJTxa3+qMn07zV2J+9a +Wdqy6x4IBFHdpfhiySeJ0ERq7JnOiRZHsTqGSF6Bg+X1S7b+cXnXyvhnEZbAomF kU8pJt6XBLvtlq5s5wqKdndqLdQGdcvHcEcEi+s4zDwQsVmYeFkSnk+uiRESp4s3 8y3ecGihNxpfl2Ro2mrrf/2qj+voZ+brs0XzQiT6yuZX1B7I7IF28Q3qwweDAjut s1G1ntQA9La615iefs2DAkdCS4ROU8/TFz30I08PNpOrwMhg6QDGGA== -----END CERTIFICATE----- clj-http-client-0.9.0/dev-resources/ssl/cert.pem000066400000000000000000000035471312026620400215550ustar00rootroot00000000000000-----BEGIN CERTIFICATE----- MIIFSjCCAzKgAwIBAgIBAzANBgkqhkiG9w0BAQsFADAfMR0wGwYDVQQDDBRQdXBw ZXQgQ0E6IGV4cGxvc2l2bzAeFw0xMzAxMDEyMTQ0MjhaFw0xODAxMDEyMTQ0Mjha MBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCC AgoCggIBANCEYBvEIdTXFOMwz18wahs6tg26C+LT2XOwQspb/Aj5WT2EBwnG2leD CzfOAKyhHDL6jRqrYU32jqoqBzmzkeVsHzqOMNFosyvcBLU8zyLZU+IP1rjJCyE8 xx9HsdhPKJj93f/gSNR5NQlRcZfqahhOwh/nYdY3pFiNgjUoRwhV2Q01n+ku8WJw kLVT1TREW9TiSWk7cHWF/ZltPOMMxvJ9q0kXh8sVYK4Gtt3pphTUW0qgXQ2NnNWT W+7vciRjnHxeoY3q6ZG7vZ8HewYKR4W8D6FA32xCsWELSsWlAABt1lBjKGas/fiY SeDqSfIxknFn/CIM9AIp2PLS3wh5e0o98qey9AN2WRyG7Qs0ijhwKx9bsMxbM0LR 5jXuXjBnjGQ69fwCjwlUsOSpNPWLibM+GmxvhghJgAlH5dD4+GqN77WLncQTWYXX GnOw5efVivS4bgU3t8l8mHLH6quLolR1KLfCv+HuqkvRposAqqLwKH+dhlbq1Y+i 4siPxfYV5NZ092Z9R0F4BPEmLhKngkK+/eQXxLY2zfaR6Ns83yRJfMXRyElECX/+ RBT1LyIRZg+MbsRg7DsKWI0plzxso/4CgSmYSfPku5nkekrMN34YhUtcxsdHSmY1 5/p2olvKpTJj3e5fa2KVswcv77FsC17gIfMXqvN3tITP+q1LLJHNAgMBAAGjgZsw gZgwNwYJYIZIAYb4QgENBCoWKFB1cHBldCBSdWJ5L09wZW5TU0wgSW50ZXJuYWwg Q2VydGlmaWNhdGUwDgYDVR0PAQH/BAQDAgWgMCAGA1UdJQEB/wQWMBQGCCsGAQUF BwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBT3awXrPXSWNszZ tlLKoXQ8LRjJ3DANBgkqhkiG9w0BAQsFAAOCAgEAYTFx++uptZxgptFmkPfT1f2W 6djOOVULmlLGPC6Ovbe5v0ksA2hbLW3eSmfL28Ku0WC8gRl0/PhyiyW77M1jp9dV ztsFkXjMiIIcY0B7Hgqh1kpK1CFvSbsD3piXcDLlZ1CwSAXuohp+J2fUblHRfAUD Th9qrm3g4uNFp0wXxO1+GgXeDrGRqYosb0wAhB7/BhW2WbOFtVYdFoyyXJFJYx/3 Gj7ZTE3rvGGxOEEuww0pFmuGCflZYxEu15Rynej7soGaE80+wRk+gMS26WKwRQ/0 TGovOHSLo/fpOjjHIoqbQLH3S08jUfAYjjP7Rd01SiztUjZMILC2WCnpdYN+2O9O rGTj2Zl3oRtE67NwxgKlo2GIFghSF366XOF8O4z1e9id6u5XEdoz8uGFkHyMu79N cdYcUtmAqLvJ0Ubewg+TfNDcfk1akNtHtIJDNqFwHlZ9R1GIupHQs10R4YxJv3I2 LojJbtcgWcDg9StwCHRA0SuLrWnHPnm+glzXM5HTNJZ6vcrM0SrcnZ59p3o3ZULL JTJikA+pcs+WAWz1yTNg/ywxYPnFrs8A4MEC43XFe2dS96a7VcNqpMMya+DAWWCS +51lDlMLuz+q5WHDGh3ouTNYMdcSLo8bHzKInDYnK/DzUpdJ5OKYwqfxv5n0bJs6 fRA2buaQo0zJeid7G2Q= -----END CERTIFICATE----- clj-http-client-0.9.0/dev-resources/ssl/key.pem000066400000000000000000000062531312026620400214050ustar00rootroot00000000000000-----BEGIN RSA PRIVATE KEY----- MIIJKQIBAAKCAgEA0IRgG8Qh1NcU4zDPXzBqGzq2DboL4tPZc7BCylv8CPlZPYQH CcbaV4MLN84ArKEcMvqNGqthTfaOqioHObOR5WwfOo4w0WizK9wEtTzPItlT4g/W uMkLITzHH0ex2E8omP3d/+BI1Hk1CVFxl+pqGE7CH+dh1jekWI2CNShHCFXZDTWf 6S7xYnCQtVPVNERb1OJJaTtwdYX9mW084wzG8n2rSReHyxVgrga23emmFNRbSqBd DY2c1ZNb7u9yJGOcfF6hjerpkbu9nwd7BgpHhbwPoUDfbEKxYQtKxaUAAG3WUGMo Zqz9+JhJ4OpJ8jGScWf8Igz0AinY8tLfCHl7Sj3yp7L0A3ZZHIbtCzSKOHArH1uw zFszQtHmNe5eMGeMZDr1/AKPCVSw5Kk09YuJsz4abG+GCEmACUfl0Pj4ao3vtYud xBNZhdcac7Dl59WK9LhuBTe3yXyYcsfqq4uiVHUot8K/4e6qS9GmiwCqovAof52G VurVj6LiyI/F9hXk1nT3Zn1HQXgE8SYuEqeCQr795BfEtjbN9pHo2zzfJEl8xdHI SUQJf/5EFPUvIhFmD4xuxGDsOwpYjSmXPGyj/gKBKZhJ8+S7meR6Ssw3fhiFS1zG x0dKZjXn+naiW8qlMmPd7l9rYpWzBy/vsWwLXuAh8xeq83e0hM/6rUsskc0CAwEA AQKCAgEAnFqGjot1KtnUab9R3/i3t49Ar+5Pt1hPd/Y1PvPWewwuJHh5ppUbZ/91 S/UGgGuWb4t8fwD/R6yXsXUuUdAamEOI7ylr5bpyj3K9fQEJ+QGpapuh0JG/7L5c OVJTQvWoZYAHFTUip1/zqvcNDHLT90InQjEIJc83RsdlWWLIULG+va72J63xhnUN rUodVsHjci/0CBpv60/7py5IN6B/FZmx9G7WfiubgSK7wI7Q4FGT2tOAswb7ERMc HoAhwLOIxaFX1sjNN1/lgffkAdV3aApqVeoFHnKr+y6ydJ2S8L9rsn+H7eXN0riu vR4xMJCHVM46O3YnCfBDI2GHqB0fmmN3GNax1MWv7NvcgT6btREaBc06IOZFtZy1 THaBHmZnoK8vTgsxaa+uRx0cPTqfoNTrrZXeBifW2h9btVV9eLICD8514SScUwmD BrOtZKKAUwSYmKV43W254Oo9JQ4GXCd1VelZBNQn/kEl5Uz48+DLSgy71sZWR8km ILtKlkMOLSOJzvcBnlpuCQzuidZSO/ORiBE11hOTLqA7re0dtIsb1MnhpiscR9gg uFWwVBiDNSFmMLj7ngPy7LAHK/ix+N3XUTuaC+17QrjR57jQUinWvpBVpZChjaVm rBjAui71JdwpkZubRmVV74QeL6UH4Tv6nS8qhJ0OnnNaGswLKYECggEBAOc9/BkH iGH+3iOIjkh/ObVDzuUr+SFIIggn0V7KEGSnDWNtMb4iceGZWwCu0q2YSY6wd40k j7TQdibzpkhMG/MFtZRlUf0XTZCYe1Lj7Ktk/TjUC1CMD6WTtmc6Khv0pTN/C+Kp fi6Uu8LYSLUJi5rsJ2IEfhh/fDFjMMVJKnf4PJ95c/Vw4Ka42jtbwt7FfSN66yRl Y69yhJuOfT+/PouoAgKwx4JjUb6JpdGNIaXtl9VUXUpGasPuF41cJZWBk5ONgTw6 XPTJOZ63WP96GssdBOyF2FFRJSs6v1wzyOb03cIA27OQ7qXyHbbxgOyhy/kF3urb 7XlB8vJwxsAtqp0CggEBAObXh62p8v3oJQ0GK95WWGvUaPDLSWTfOFJpoUBV45Bi ATdTIIBAbqTXgpj42w5iyvu/61p9lK5U5nUBb+VYpKrB0MlmelizuVLH27jpQXXz TYAv2rgfQqgF4OpvmaoMK/BmHofQjTYCm9s7X2FR4hLMqUpdb5SeybvA6K95oWGI 3wwk3ThOypwTd1Pi/6C/7WiDrCO3TWU1UmAneHVxUm+9iBGyZGxxJOFcxE+GEetC uA76jURnvmC6k5nbJy1+/SwtRJhOWtVyE/xY846KF1PnZzCFVVLvRXxNZB1VoFd5 61/3vF3KixrQC8q1gly4Zqeo+fZnQOIUyE2/ARwuhPECggEAaqg4YwMKcMixhQoz NlUYNPc1spZ5rlQq/j//Xg0tSn+SuU1gKCaTCE9HniUEn3UiWGIkgkFe1zNfi8/N 3oLcUVdMzUl+a5IYAJ2UJENkohlOgqurHFe9z8010J8PVR2eJQZwYPd0b9/CSrif sIDal3ZdI+SWlI4Ypl7t29FHeVZR/+xxA8AwnjWc0swcMcw9T+QeGQd61y2m5Gjw dDqtipTPeJY5L3bH+W5bwS+rWXEhGxByhxO7outqiZT68N53RxN5jGIPBgjaPs6q iguz2ANmhgGmKLuYvTu3j7uC6qP+tMDYlRZAPOJTulHh+UMXZaDDlOgjvE4i1JgX 1AQ8MQKCAQAwFxSP7EjP9o3JcdCvyAMxq5WVHHSUzB/6o0DRm1MGIDSqpumtbj7e nnr5jVZtX81ztt68Ak94Jf3AwGTkPZxIaoopeuzgD8j64uH2WrmlbeNmYhHJq9GC GX9qt4cmstRwh4Wyu5K/frmjaXIMXzeevP5DnMWDC7VxJNYUwF0Laa015XQkp91z uGZylma0wWcfD9dLtYMtI3eeynpA1TPcTXrMXQKoyMVrIZ1QB9kxzrtze2T0rDww AI6BTfOSedMaYe/ZXvFzaAmb15gdyMzlUN7hb9V/qpMqOyExL0ZxEtgjLQQT8f8a vd7HAxs+X7gbE4vHdmlA8B1ufO0pRtOxAoIBAQCKPpWBiM09HRBlS4WbOOM+BZCd +15VO4bC+j8y/4R2TKUVTlRQW0aFwS6wiNBpSHsh20LESI9vlabOG2X7kJCc17QS LswdMmu7YF5HRgW1OeJhUkywbqeZJGdD0uBBKZy0zuygBVJQDyXLI0GukSjJITW9 j6k6EKDLM5JMmydKih27ObJ1ztDRiGVZaQBX4XqEFxW8AFRIFm3c4Tvz9bxZbC87 fKwayKfc5ClVa/8eQJLvLLWJ1fnb8YyAUYXFUMXfv3AN5cu+Y/KmbIY/6CYd77y8 RQaRtpg14BH7su8Ynjzr0TRB88O6XXFMf59sa5qTCx8Q88JwdrygQcc2jccI -----END RSA PRIVATE KEY----- clj-http-client-0.9.0/doc/000077500000000000000000000000001312026620400152625ustar00rootroot00000000000000clj-http-client-0.9.0/doc/clojure-client.md000066400000000000000000000205621312026620400205300ustar00rootroot00000000000000## Making requests with clojure clients clj-http-client allows you to make requests in two ways with clojure clients: with and without a persistent HTTP client. ## `create-client` clj-http-client allows you to create a persistent synchronous or asynchronous HTTP client using the `create-client` function from the corresponding namespace. The `create-client` function takes one argument, a map called `options`. The available options for configuring the client are detailed below. ### Base Options The following are the base set of options supported by the `create-client` functions. * `:force-redirects`: used to set whether or not the client should follow redirects on POST or PUT requests. Defaults to false. * `:follow-redirects`: used to set whether or not the client should follow redirects in general. Defaults to true. If set to false, will override the :force-redirects setting. * `:connect-timeout-milliseconds`: maximum number of milliseconds that the client will wait for a connection to be established. A value of 0 is interpreted as infinite. A negative value for or the absence of this option is interpreted as undefined (system default). * `:socket-timeout-milliseconds`: maximum number of milliseconds that the client will allow for no data to be available on the socket before closing the underlying connection, 'SO_TIMEOUT' in socket terms. A timeout of zero is interpreted as an infinite timeout. A negative value for or the absence of this setting is interpreted as undefined (system default). * `:ssl-protocols`: an array used to set the list of SSL protocols that the client could select from when talking to the server. Defaults to 'TLSv1', 'TLSv1.1', and 'TLSv1.2'. * `:cipher-suites`: an array used to set the cipher suites that the client could select from when talking to the server. Defaults to the complete set of suites supported by the underlying language runtime. * `:metric-registry`: a Dropwizard `MetricRegistry` to register http metrics to. If provided, metrics will automatically be registered for all requests made by the client. See the [metrics documentation](./metrics.md) for more info. * `:server-id`: a string for the name of the server the request is being made from. If specified, used in the namespace for metrics: `puppetlabs..http-client.experimental`. * `:metric-prefix`: a string for the prefix for metrics. If specified, metric namespace is `.http-client.experimental`. If both `metric-prefix` and `server-id` are specified, `metric-prefix` takes precendence. ### SSL Options The following options are SSL specific, and only one of the following combinations is permitted. * `:ssl-context`: an instance of [SSLContext](http://docs.oracle.com/javase/7/docs/api/javax/net/ssl/SSLContext.html) OR * `:ssl-cert`: path to a PEM file containing the client cert * `:ssl-key`: path to a PEM file containing the client private key * `:ssl-ca-cert`: path to a PEM file containing the CA cert OR * `:ssl-ca-cert`: path to a PEM file containing the CA cert ### Making requests with a persistent client The `create-client` functions return a client with the following protocol: ```clj (defprotocol HTTPClient (get [this url] [this url opts]) (head [this url] [this url opts]) (post [this url] [this url opts]) (put [this url] [this url opts]) (delete [this url] [this url opts]) (trace [this url] [this url opts]) (options [this url] [this url opts]) (patch [this url] [this url opts]) (close [this])) ``` Each function will execute the corresponding HTTP request, with the exception of `close`, which will close the client. Each request function takes one argument, `url`, which is the URL against which you want to make your HTTP request. Each request function also has a two-arity version with an extra parameter, `options`, which is a map containing options for the HTTP request. These options are as follows: * `:headers`: optional; a map of headers. By default an 'accept-language' header with a value of `puppetlabs.core.i18n/user-locale` will be added to the request. * `:body`: optional; may be a String or any type supported by clojure's reader * `:compress-request-body`: optional; used to control any additional compression which the client can apply to the request body before it is sent to the target server. Defaults to `:none`. Supported values are: * `:gzip` which will compress the request body as gzip * `:none` which will not apply any additional compression to the request body * `:decompress-body`: optional; if `true`, an 'accept-encoding' header with a value of 'gzip, deflate' will be added to the request, and the response will be automatically decompressed if it contains a recognized 'content-encoding' header. Defaults to `true`. * `:as`: optional; used to control the data type of the response body. Defaults to `:stream`. Supported values are: * `:text` which will return a `String` * `:stream` which will return an `InputStream` * `:unbuffered-stream` which is a variant of `:stream` that will buffer as little data as possible * `:query-params`: optional; used to set the query parameters of an http request. This should be a map, where each key and each value is a String. * `:metric-id`: optional; a vector of keywords or strings. A metric will be created for each element in the vector, with each appending to the previous. For example, say you want to make a GET request with query parameter `abc` with value `def` to the URL `http://localhost:8080/test`. If you wanted to use a persistent synchronous client, you could make the request and print the body of the response like so: ```clj (let [client (sync/create-client {}) response (get client "http://localhost:8080/test" {:query-params {"abc" "def"}})] (println (:body response)) ``` If you wanted to use an asynchronous client, you could make the request and print the body of the response like so: ```clj (let [client (async/create-client {}) response (get client "http://localhost:8080/test" {:query-params {"abc" "def"}})] (println (:body @response))) ``` ### Closing a persistent client The `close` function takes no arguments. This function closes the client, and causes all resources associated with it to be cleaned up. This function must be called by the caller when they are done making requests with the client, as no implicit cleanup of the associated resources is done when the client is garbage collected. Once a client is closed, it can no longer be used to make any requests. ## Making a Request without a persistent client In addition to allowing you to create a persistent client with the `create-client` function, the puppetlabs.http.client.sync namespace provides the following simple request functions that can be called without a client: ```clj (get [url] [url opts]) (head [url] [url opts]) (post [url] [url opts]) (put [url] [url opts]) (delete [url] [url opts]) (trace [url] [url opts]) (options [url] [url opts]) (patch [url] [url opts]) (request [req]) ``` These functions will, for every request, create a new client, make a new request with that client, and then close the client once the response is received. Each of these functions (barring `request`) take one argument, `url`, which is the URL to which you want to make the request, and can optionally take a second argument, `options`. `options` is a map of options to configure both the client and the request, and as such takes the union of all options accepted by the `create-client` function and all options accepted by the request functions for a persistent client. For example, say you want to make a GET request to the URL `http://localhost:8080/test` with query parameter `abc` with value `def`, and you do not want redirects to be followed. In that case, you could do the following to make the request and print the body of the response: ```clj (let [response (get "http://localhost:8080/test" {:follow-redirects false :query-params {"abc" "def"}})] (println (:body response))) ``` A `request` function is also provided, which allows you to make a request of any type. `request` takes one argument, `req`, which is a map of options. It takes the same options as the simple request functions, but also takes the following required options: * `:url`: the URL against which to make the request. This should be a string. * `:method`: the HTTP method (:get, :head, :post, :put, :delete, :trace, :options, :patch) clj-http-client-0.9.0/doc/java-client.md000066400000000000000000000074301312026620400200050ustar00rootroot00000000000000## Making requests with the Java client Similarly to the way it is done in clojure code, clj-http-client allows you to make requests in two ways using Java: with and without a persistent client. ## `createClient(ClientOptions clientOptions)` clj-http-client allows you to create a persistent synchronous or asynchronous HTTP client using the static `createClient()` method in the [`Async`](../src/java/com/puppetlabs/http/client/Async.java) and [`Sync`](../src/java/com/puppetlabs/http/client/Sync.java) classes This method takes one argument, `clientOptions`, which is an instance of the [`ClientOptions`](../src/java/com/puppetlabs/http/client/ClientOptions.java) class, details on which can be found in its javadoc strings, linked above. ### Making requests with a persistent client The `createClient()` method returns an object implementing the [`SyncHttpClient`](../src/java/com/puppetlabs/http/client/SyncHttpClient.java) interface in the case of `Sync`, and the [`AsyncHttpClient`](../src/java/com/puppetlabs/http/client/AsyncHttpClient.java) interface in the case of `Async`. Information on the various methods available is detailed in the javadoc strings for the corresponding interfaces, which are linked above. The various request methods provided by these interfaces can take a [`RequestOptions`](../src/java/com/puppetlabs/http/client/RequestOptions.java) object, information on which can be found in that class' javadoc strings, linked above. For example, say you want to make a GET request against the URL `http://localhost:8080/test` with query parameter `abc` with value `def`. To make the request and print the body of the response with a persistent synchronous client, you could do the following: ```java ClientOptions options = new ClientOptions(); SyncHttpClient client = Sync.createClient(options); Response response = client.get(new URI("http://localhost:8080/test?abc=def")); System.out.println(response.getBody()); ``` If instead you wanted to use an asynchronous client, you could do the following: ```java ClientOptions options = new ClientOptions(); AsyncHttpClient client = Async.createClient(options); Promise response = client.get(new URI("http://localhost:8080/test?abc=def")); System.out.println(response.deref().getBody()); ``` ### Closing the client Each persistent client provides a `close` method, which can be used to close the client. This method will close the client and clean up all resources associated with it. It must be called by the caller when finished using the client to make requests, as there is no implicit cleanup of the associated resources when the client is garbage collected. Once the client is closed, it can no longer be used to make requests. ## Making a Request without a persistent client In addition to allowing you to create a persistent client with the `createClient()` method, the [`Sync`](../src/java/com/puppetlabs/http/client/Sync.java) class contains a number of simple request methods that allow for requests to be made without a persistent client. These are detailed in `Sync.java`'s javadoc strings, linked above. Many of the provided request methods take a [`SimpleRequestOptions`](../src/java/com/puppetlabs/http/client/SimpleRequestOptions.java) object. Information on this class can be found in its javadoc strings, linked above. As an example, say you wanted to make a request to the URL `http://localhost:8080/test` without a persistent client. You want the query parameter `abc` with value `def`, and you don't want redirects to be followed. In that case, you would do the following to print the body of the response: ```java SimpleRequestOptions options = new SimpleRequestOptions(new URI("http://localhost:8080/test?abc=def")); options = options.setFollowRedirects(false); Response response = Sync.get(options); System.out.println(response.getBody()); ```clj-http-client-0.9.0/doc/metrics.md000066400000000000000000000424421312026620400172600ustar00rootroot00000000000000# Metrics Both the Java client and the Clojure client have [Dropwizard Metrics](http://metrics.dropwizard.io/3.1.0/) support - they both accept as an option a `MetricRegistry` to which they will register http metrics for each request, as well as metrics for any metric-ids specified in the request options. This support is experimental - names of metrics and the exact API may change. For using metrics with either the Java client or the Clojure client you must already have created a Dropwizard `MetricRegistry`. - [Metric namespace](#metric-namespace) - [Types of metrics](#types-of-metrics) - [Getting back metrics](#getting-back-metrics) - [Clojure API](#clojure-api) - [Creating a client with metrics](#creating-a-client-with-metrics) - [Setting a metric-id](#setting-a-metric-id) - [Filtering metrics](#filtering-metrics) - [Filtering by URL](#filtering-by-url) - [Filtering by URL and method](#filtering-by-url-and-method) - [Filtering by metric-id](#filtering-by-metric-id) - [Java API](#java-api) - [Creating a client with metrics](#creating-a-client-with-metrics-1) - [Setting a metric-id](#setting-a-metric-id-1) - [Filtering metrics](#filtering-metrics-1) - [Filtering by URL](#filtering-by-url-1) - [Filtering by URL and method](#filtering-by-url-and-method-1) - [Filtering by metric-id](#filtering-by-metric-id-1) ## Metric namespace By default, http metrics are prefixed with the namespace `puppetlabs.http-client.experimental`. This namespace can be customized with two client options, `server-id` and `metric-prefix`. When `server-id` is set, the metric namespace becomes `puppetlabs..http-client.experimental`. When `metric-prefix` is set, the metric namespace becomes `.http-client.experimental`. If both `server-id` and `metric-prefix` are set, `metric-prefix` wins out and a warning message is logged. For a Clojure client, the `get-client-metric-namespace` protocol method can be used to get back the metric namespace set for the client. For a Java client, the `getMetricNamespace` method can be used to get back the configured metric namespace. ## Types of metrics There are two types of metrics: *full response* and *initial response*. Full response metrics stop when all bytes of the response have been read by the client. Initial response metrics stop when the first byte of the response has been received by the client. Full response metrics are suffixed with `full-response`. Initial response metric support has not yet been implementedare suffixed with `full-response`. Initial response metric support has not yet been implemented. There are three categories of metrics: `url` metrics, `url-and-method` metrics, and `metric-id` metrics. `url` and `url-and-method` metrics are created automatically for each request; `metric-id` metrics are only created for requests that have a `metric-id` request option specified. Each http request will have a metric name created for its url (stripped of query strings and url fragments), as well as for the url + method name. `url` metrics have `with-url` after the prefix, followed by the url. `url-and-method` metrics have `with-url-and-method` after the prefix, followed by the url and then the capitalized HTTP method. So, for example, a `GET` request to `http://foobar.com` would create a metric `puppetlabs.http-client.experimental.with-url.http://foobar.com.full-response` and a metric `puppetlabs.http-client.experimental.with-url-and-method.http://foobar.com.GET.full-response`. It is also possible to give a request a `metric-id`. The `metric-id` is an array of strings. For each element in the array, a metric name will be created, appending to the previous elements. `metric-id` metrics have `with-metric-id` after the metric prefix, followed by the metric-id. So, for example, for a metric-id `["foo", "bar", "baz"]`, the metrics `puppetlabs.http-client.experimental.with-metric-id.foo.full-response`, `puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response`, and `puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response` would be created. ## Getting back metrics Both the Clojure API and the Java API have functions to get back from a `MetricRegistry` either the `Timer` objects or a selection of metric data, and to filter this information based on url, url and method, or metric-id. There are three different `Timer` objects: `UrlClientTimer` (which includes a field for the url and `getUrl` method), `UrlAndMethodClientTimer` (which includes a field for the url and a field for the method and accompanying `getUrl` and `getMethod` methods), and `MetricIdClientTimer` (which includes a field for the metric-id and accompanying `getMetricId` method). Each of these timers also has an `isCategory` method that returns whether the timer is or is not the provided `MetricCategory`. Both the Clojure and Java APIs also include functions that return a list of metrics data. This metrics data includes the `metric-name`, `count`, `mean` (in ms), and `aggregate` (computed as `count * mean`) for each metric. In addition, for `url` metrics the accompanying metrics data includes the `url`, for `url-and-method` metrics it includes the `url` and `method`, and for `metric-id` metrics it includes the `metric-id`. The Java API returns `ClientMetricData` objects - of which there are three types - `UrlClientMetricData`, `UrlAndMethodClientMetricData`, and `MetricIdClientMetricData`. The Clojure API returns maps with keys for this information. Both APIs have functions for returning all metrics/metric data, for returning all metrics/metrics data from a specific category, and for filtering metrics/metrics data within a category. ## Clojure API ### Creating a client with metrics To use metrics with the Clojure client, pass a `MetricRegistry` to it as part of the options map: ```clojure (async/create-client {:metric-registry (MetricRegistry.)}) ``` (the same works with the `sync` client). In [Trapperkeeper](https://github.com/puppetlabs/trapperkeeper) applications, creating and managing a `MetricRegistry` can be done easily with [trapperkeeper-metrics](https://github.com/puppetlabs/trapperkeeper-metrics): ```clojure (defservice my-trapperkeeper-service MyService [[:MetricsService get-metrics-registry]] (init [this context] (let [registry (get-metrics-registry) client (async/create-client {:metric-registry registry})] ...))) ``` Any client that is created with a `MetricRegistry` will automatically have `url` and `url-and-method` metrics registered. `get-client-metric-registry` protocol function can be called on a client to get the `MetricRegistry` from it. ### Setting a metric-id In addition to the `url` and `url-and-method` metrics, it is possible to set a `metric-id` for a request that will create additional metrics. For the Clojure API, a `metric-id` is a vector of keywords or strings. Either is supported by the API, however, if special characters are needed, use strings. Note than even when specified as a vector of keywords in the request option, the metric-id will be returned as a vector of strings in the metrics data. To set a `metric-id` for a request, include it as an option in the request options map. ```clojure (common/get client "https://foobar.com" {:metric-id [:foo :bar :baz]}) (common/get client "https://foobar.com" {:metric-id ["f/o/o"]}) ``` ### Filtering metrics To get all `Timer` objects registered for a `MetricRegistry`, use the `get-client-metrics` function in the `metrics` namespace. This takes the `MetricRegistry` as an argument and returns a map with three keys: `:url` `:url-and-method`, and `:metric-id`. Under each of these keys is a sequence of `Timer` objects of the corresponding type. If there are no timers of a certain type, the sequence will be empty. The output of this function conforms to the `common/AllTimers` schema. Example: ```clojure (common/get client "http://test.com" {:metric-id [:foo :bar :baz]}) ... (metrics/get-client-metrics metric-registry) => {:url [#object[com.puppetlabs.http.client.metrics.UrlClientTimer 0x66cf1f05 "com.puppetlabs.http.client.metrics.UrlClientTimer@66cf1f05"]] :url-and-method [#object[com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer 0x6fe5444c "com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer@6fe5444c"]] :metric-id [#object[com.puppetlabs.http.client.metrics.MetricIdClientTimer 0x690c10c5 "com.puppetlabs.http.client.metrics.MetricIdClientTimer@690c10c5"] #object[com.puppetlabs.http.client.metrics.MetricIdClientTimer 0xb7aca2e "com.puppetlabs.http.client.metrics.MetricIdClientTimer@b7aca2e"] #object[com.puppetlabs.http.client.metrics.MetricIdClientTimer 0x4ef82829 "com.puppetlabs.http.client.metrics.MetricIdClientTimer@4ef82829"]]} ``` To get metric data for all `Timer`s registered on a `MetricRegistry`, use the `get-client-metrics-data` function. This takes the `MetricRegistry` and returns a map with `:url`, `:url-and-method`, and `:metric-id` as keys. Under each of these keys is a sequence of maps, each map containing metrics data (see [Getting back metrics](#Getting-back-metrics) above, conforming to the `common/AllMetricsData` schema. Example: ```clojure (common/get client "http://test.com" {:metric-id [:foo :bar :baz]}) ... (metrics/get-client-metrics-data metric-registry) => {:url ({:count 1 :mean 553 :aggregate 553 :metric-name "puppetlabs.http-client.experimental.with-url.http://test.com.full-response" :url "http://test.com"}) :url-and-method ({:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-url-and-method.http://test.com.GET.full-response" :url "http://test.com" :method "GET"}) :metric-id ({:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response" :metric-id ["foo" "bar" "baz"]} {:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response" :metric-id ["foo" "bar"]} {:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.full-response" :metric-id ["foo"]})} ``` #### Filtering by URL To get URL metrics and metrics data, use the `get-client-metrics-by-url` function and `get-client-metrics-data-by-url` in the `metrics` namespace. Both of these take as arguments the `MetricRegistry` and a string url. If no url is provided, return all url metrics/metrics data in a sequence. If no metrics are registered for that url, return an empty sequence. Example: ```clojure (common/get client "http://test.com" {:metric-id [:foo :bar :baz]}) ... (metrics/get-client-metrics-data-by-url metric-registry "http://test.com") => ({:count 1 :mean 553 :aggregate 553 :metric-name "puppetlabs.http-client.experimental.with-url.http://test.com.full-response" :url "http://test.com"}) (metrics/get-client-metrics-data-by-url metric-registry "http://not-a-matching-url.com") => () ``` #### Filtering by URL and method To get URL and method metrics and metrics data, use the `get-client-metrics-by-url-and-method` and `get-client-metrics-data-by-url-and-method` functions in the `metrics` namespace. Both of these take as arguments the `MetricRegistry`, a string url, and a keyword HTTP method. If no url and method is provided, return all url metrics/metrics data in a sequence. If no metrics are registered for that url and method, return an empty sequence. Example: ```clojure (common/get client "http://test.com" {:metric-id [:foo :bar :baz]}) ... (metrics/get-client-metrics-data-by-url-and-method metric-registry "http://test.com" :get) => ({:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-url-and-method.http://test.com.GET.full-response" :url "http://test.com" :method "GET"}) :method "GET"}) (metrics/get-client-metrics-data-by-url-and-method metric-registry "http://test.com" :post) => () ``` #### Filtering by metric-id To get metric-id metrics and metrics data, use the `get-client-metrics-by-metric-id` and `get-client-metrics-data-by-metric-id` functions in the `metrics` namespace. Both of these take as arguments the `MetricRegistry` and a metric-id - as a vector of keywords or strings (if special characters are needed, use strings). If no metric-id is provided, will return all metric-id metrics. If no metrics are registered for that metric-id, returns an empty list. Example: ```clojure (common/get client "http://test.com" {:metric-id [:foo :bar :baz]}) ... (metrics/get-client-metrics-data-by-metric-id metric-registry [:foo]) => ({:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.full-response" :metric-id ["foo"]}) (metrics/get-client-metrics-data-by-metric-id metric-registry [:foo :bar]) => ({:count 1 :mean 554 :aggregate 554 :metric-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response" :metric-id ["foo" "bar"]}) (metrics/get-client-metrics-data-by-metric-id metric-registry [:foo :nope]) => () ``` ## Java API ### Creating a client with metrics To use metrics with the Java client, call `setMetricRegistry()` on the `ClientOptions` before supplying them to create the client. ```java MetricRegistry registry = new MetricRegistry(); ClientOptions options = new ClientOptions(); AsyncHttpClient client = Async.createClient(options); ``` (the same works with the `SyncHttpClient`). Any client that is created with a `MetricRegistry` will automatically have `url` and `url-and-method` metrics registered. `getMetricRegistry()` can be called on a client to get the `MetricRegistry` from it. ### Setting a metric-id In addition to the `url` and `url-and-method` metrics, it is possible to set a `metric-id` for a request that will create additional metrics. A `metric-id` is an array of strings. To set a `metric-id` for a request, use the `.setMetricId` method on the `RequestOptions` class. ```java RequestOptions options = new RequestOptions("https://foobar.com"); options.setMetricId(["foo", "bar", "baz"]); client.get(options); ``` `getMetricId()` can be called on `RequestOptions` to get back the `metric-id`. ### Filtering metrics To get all `Timer` objects registered for a `MetricRegistry`, use the `getClientMetrics()` method in the `Metrics` class. This takes the `MetricRegistry` as an argument and returns a `ClientTimerContainer` object. The `ClientTimerContainer` object has three fields - `urlTimers`, `urlAndMethodTimers`, and `metricIdTimers`. The list of `URLClientTimers` can be retrieved from the `ClientTimerContainer` with the `getUrlTimers()` method. A list of `URlAndMethodClientTimers` can be retrieved with the `getUrlAndMethod()` method. A list of `MetricIdClientTimer`s can be retrieved with the `getMetricIdTimers()` method. To get all `MetricData` objects, representing data for each `Metric` registered for a `MetricRegistry`, use the `getClientMetricsData()` methods in the `Metric` class. This takes the `MetricRegistry` as an argument and returns a `ClientMetricDataContainer` object. This object has a field for each type of metric data - `urlData`, `urlAndMethodData`, and `metricIdData`. Each field has an associated getter returning a list of the appropriate data object - `UrlClientMetricData`, `UrlAndMethodClientMetricData`, and `MetricIdClientMetricData`. #### Filtering by URL To get URL metrics and metrics data, use the `getClientMetricsByUrl()` and `getClientMetricsDataByUrl()` methods in the `Metric` class. Both of these take as arguments the `MetricRegistry` and a string url. If no url is provided, return all url metrics. If no metrics are registered for that url, return an empty list. `getClientMetricsByUrl()` returns a list of `UrlClientTimer` objects. `getClientMetricsDataByUrl` returns a list of `UrlClientMetricData` objects. #### Filtering by URL and method To get URL and method metrics and metrics data, use the `getClientMetricsByUrlAndMethod()` and `getClientMetricsDataByUrlAndMethod()` methods in the `Metric` class. Both of these take as arguments the `MetricRegistry`, a string url, and a string HTTP method. If no url or method is provided, will return all url-and-method metrics. If no metrics are registered for that url and method, returns an empty list. `getClientMetricsByUrlAndMethod()` returns a list of `UrlAndMethodClientTimer` objects. `getClientMetricsDataByUrlAndMethod` returns a list of `UrlAndMethodClientMetricData` objects. #### Filtering by metric-id To get metric-id metrics and metrics data, use the `getClientMetricsByMetricId()` and `getClientMetricsDataByMetricId()` methods in the `Metric` class. Both of these take as arguments the `MetricRegistry` and a metric-id - as an array of strings. If no metric-id is provided, will return all metric-id metrics. If no metrics are registered for that metric-id, returns an empty list. `getClientMetricsByMetricId()` returns a list of `MetricIdClientTimer` objects. `getClientMetricsDataByMetricId()` returns a list of `MetricIdClientMetricData` objects. clj-http-client-0.9.0/ext/000077500000000000000000000000001312026620400153155ustar00rootroot00000000000000clj-http-client-0.9.0/ext/travisci/000077500000000000000000000000001312026620400171415ustar00rootroot00000000000000clj-http-client-0.9.0/ext/travisci/test.sh000077500000000000000000000000451312026620400204560ustar00rootroot00000000000000#!/bin/bash set -e lein2 test :all clj-http-client-0.9.0/jenkins/000077500000000000000000000000001312026620400161565ustar00rootroot00000000000000clj-http-client-0.9.0/jenkins/deploy.sh000077500000000000000000000003361312026620400200130ustar00rootroot00000000000000#!/usr/bin/env bash set -e set -x git fetch --tags lein test echo "Tests passed!" lein release echo "Release plugin successful, pushing changes to git" git push origin --tags HEAD:$BRANCH echo "git push successful." clj-http-client-0.9.0/locales/000077500000000000000000000000001312026620400161375ustar00rootroot00000000000000clj-http-client-0.9.0/locales/eo.po000066400000000000000000000013151312026620400171020ustar00rootroot00000000000000# Esperanto translations for puppetlabs.http_client package. # Copyright (C) 2017 Puppet # This file is distributed under the same license as the puppetlabs.http_client package. # Automatically generated, 2017. # msgid "" msgstr "" "Project-Id-Version: puppetlabs.http_client \n" "Report-Msgid-Bugs-To: docs@puppet.com\n" "POT-Creation-Date: \n" "PO-Revision-Date: \n" "Last-Translator: Automatically generated\n" "Language-Team: none\n" "Language: eo\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" "Plural-Forms: nplurals=2; plural=(n != 1);\n" #: src/clj/puppetlabs/http/client/async.clj msgid "Unsupported request method: {0}" msgstr "" clj-http-client-0.9.0/locales/http-client.pot000066400000000000000000000012561312026620400211220ustar00rootroot00000000000000# SOME DESCRIPTIVE TITLE. # Copyright (C) YEAR Puppet # This file is distributed under the same license as the puppetlabs.http_client package. # FIRST AUTHOR , YEAR. # #, fuzzy msgid "" msgstr "" "Project-Id-Version: puppetlabs.http_client \n" "Report-Msgid-Bugs-To: docs@puppet.com\n" "POT-Creation-Date: \n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language-Team: LANGUAGE \n" "Language: \n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=UTF-8\n" "Content-Transfer-Encoding: 8bit\n" #: src/clj/puppetlabs/http/client/async.clj msgid "Unsupported request method: {0}" msgstr "" clj-http-client-0.9.0/project.clj000066400000000000000000000050031312026620400166530ustar00rootroot00000000000000(defproject puppetlabs/http-client "0.9.0" :description "HTTP client wrapper" :license {:name "Apache License, Version 2.0" :url "http://www.apache.org/licenses/LICENSE-2.0.html"} :min-lein-version "2.7.1" :parent-project {:coords [puppetlabs/clj-parent "0.6.1"] :inherit [:managed-dependencies]} ;; Abort when version ranges or version conflicts are detected in ;; dependencies. Also supports :warn to simply emit warnings. ;; requires lein 2.2.0+. :pedantic? :abort :dependencies [[org.clojure/clojure] [org.apache.httpcomponents/httpasyncclient "4.1.2"] [prismatic/schema] [commons-io] [io.dropwizard.metrics/metrics-core "3.1.2"] [puppetlabs/ssl-utils] [puppetlabs/i18n]] :source-paths ["src/clj"] :java-source-paths ["src/java"] :jar-exclusions [#".*\.java$"] ;; By declaring a classifier here and a corresponding profile below we'll get an additional jar ;; during `lein jar` that has all the source code (including the java source). Downstream projects can then ;; depend on this source jar using a :classifier in their :dependencies. :classifiers [["sources" :sources-jar]] :profiles {:dev {:dependencies [[cheshire] [puppetlabs/kitchensink nil :classifier "test"] [puppetlabs/trapperkeeper] [puppetlabs/trapperkeeper nil :classifier "test"] [puppetlabs/trapperkeeper-webserver-jetty9] [puppetlabs/trapperkeeper-webserver-jetty9 nil :classifier "test"] [puppetlabs/ring-middleware]] ;; TK-143, enable SSLv3 for unit tests that exercise SSLv3 :jvm-opts ["-Djava.security.properties=./dev-resources/java.security"]} :sources-jar {:java-source-paths ^:replace [] :jar-exclusions ^:replace [] :source-paths ^:replace ["src/clj" "src/java"]}} :deploy-repositories [["releases" {:url "https://clojars.org/repo" :username :env/clojars_jenkins_username :password :env/clojars_jenkins_password :sign-releases false}]] :lein-release {:scm :git :deploy-via :lein-deploy} :plugins [[lein-parent "0.3.1"] [puppetlabs/i18n "0.8.0"]]) clj-http-client-0.9.0/src/000077500000000000000000000000001312026620400153045ustar00rootroot00000000000000clj-http-client-0.9.0/src/clj/000077500000000000000000000000001312026620400160545ustar00rootroot00000000000000clj-http-client-0.9.0/src/clj/puppetlabs/000077500000000000000000000000001312026620400202335ustar00rootroot00000000000000clj-http-client-0.9.0/src/clj/puppetlabs/http/000077500000000000000000000000001312026620400212125ustar00rootroot00000000000000clj-http-client-0.9.0/src/clj/puppetlabs/http/client/000077500000000000000000000000001312026620400224705ustar00rootroot00000000000000clj-http-client-0.9.0/src/clj/puppetlabs/http/client/async.clj000066400000000000000000000321051312026620400243000ustar00rootroot00000000000000;; This namespace is a wrapper around the http client functionality provided ;; by Apache HttpAsyncClient. It allows the use of PEM files for HTTPS configuration. ;; ;; In the options to any request method, an existing SSLContext object can be ;; supplied under :ssl-context. If this is present it will be used. If it's ;; not, the wrapper will attempt to use a set of PEM files stored in ;; (:ssl-cert :ssl-key :ssl-ca-cert) to create the SSLContext. ;; ;; See the puppetlabs.http.sync namespace for synchronous versions of all ;; these methods. (ns puppetlabs.http.client.async (:import (com.puppetlabs.http.client ClientOptions RequestOptions ResponseBodyType HttpMethod CompressType) (com.puppetlabs.http.client.impl JavaClient ResponseDeliveryDelegate) (org.apache.http.client.utils URIBuilder) (org.apache.http.nio.client HttpAsyncClient) (com.codahale.metrics MetricRegistry) (java.util Locale)) (:require [puppetlabs.http.client.common :as common] [schema.core :as schema] [puppetlabs.http.client.metrics :as metrics] [puppetlabs.i18n.core :as i18n :refer [trs]] [clojure.string :as str]) (:refer-clojure :exclude (get))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Private utility functions (schema/defn ^:always-validate create-default-client :- HttpAsyncClient [{:keys [ssl-context ssl-ca-cert ssl-cert ssl-key ssl-protocols cipher-suites follow-redirects force-redirects connect-timeout-milliseconds socket-timeout-milliseconds metric-registry server-id metric-prefix enable-url-metrics?]}:- common/ClientOptions] (let [client-options (ClientOptions.)] (cond-> client-options (some? ssl-context) (.setSslContext ssl-context) (some? ssl-cert) (.setSslCert ssl-cert) (some? ssl-ca-cert) (.setSslCaCert ssl-ca-cert) (some? ssl-key) (.setSslKey ssl-key) (some? ssl-protocols) (.setSslProtocols (into-array String ssl-protocols)) (some? cipher-suites) (.setSslCipherSuites (into-array String cipher-suites)) (some? force-redirects) (.setForceRedirects force-redirects) (some? follow-redirects) (.setFollowRedirects follow-redirects) (some? connect-timeout-milliseconds) (.setConnectTimeoutMilliseconds connect-timeout-milliseconds) (some? socket-timeout-milliseconds) (.setSocketTimeoutMilliseconds socket-timeout-milliseconds) (some? metric-registry) (.setMetricRegistry metric-registry) (some? server-id) (.setServerId server-id) (some? metric-prefix) (.setMetricPrefix metric-prefix) (some? enable-url-metrics?) (.setEnableURLMetrics enable-url-metrics?)) (JavaClient/createClient client-options))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Map the Ring request onto the Java API (schema/defn callback-response :- common/Response [opts :- common/UserRequestOptions callback :- common/ResponseCallbackFn response :- common/Response] (if callback (try (callback response) (catch Exception e {:opts opts :error e})) response)) (schema/defn java-content-type->clj :- common/ContentType [java-content-type] (if java-content-type {:mime-type (.getMimeType java-content-type) :charset (.getCharset java-content-type)})) (schema/defn get-response-delivery-delegate :- ResponseDeliveryDelegate [opts :- common/UserRequestOptions result :- common/ResponsePromise] (reify ResponseDeliveryDelegate (deliverResponse [_ _ orig-encoding body headers status content-type callback] (->> {:opts opts :orig-content-encoding orig-encoding :status status :headers (into {} headers) :content-type (java-content-type->clj content-type) :body body} (callback-response opts callback) (deliver result))) (deliverResponse [_ _ e callback] (->> {:opts opts :error e} (callback-response opts callback) (deliver result))))) (schema/defn clojure-method->java [opts :- common/UserRequestOptions] (case (:method opts) :delete HttpMethod/DELETE :get HttpMethod/GET :head HttpMethod/HEAD :options HttpMethod/OPTIONS :patch HttpMethod/PATCH :post HttpMethod/POST :put HttpMethod/PUT :trace HttpMethod/TRACE (throw (IllegalArgumentException. (trs "Unsupported request method: {0}" (:method opts)))))) (defn- parse-url [{:keys [url query-params]}] (if (nil? query-params) url (let [uri-builder (reduce #(.addParameter %1 (key %2) (val %2)) (.clearParameters (URIBuilder. url)) query-params)] (.build uri-builder)))) (schema/defn clojure-response-body-type->java :- ResponseBodyType [opts :- common/RequestOptions] (case (:as opts) :unbuffered-stream ResponseBodyType/UNBUFFERED_STREAM :text ResponseBodyType/TEXT ResponseBodyType/STREAM)) (schema/defn clojure-compress-request-body-type->java :- CompressType [opts :- common/RequestOptions] (case (:compress-request-body opts) :gzip CompressType/GZIP CompressType/NONE)) (defn parse-metric-id [opts] (when-let [metric-id (:metric-id opts)] (into-array (map name metric-id)))) (schema/defn clojure-options->java :- RequestOptions [opts :- common/RequestOptions] (-> (parse-url opts) RequestOptions. (.setAs (clojure-response-body-type->java opts)) (.setBody (:body opts)) (.setDecompressBody (clojure.core/get opts :decompress-body true)) (.setCompressRequestBody (clojure-compress-request-body-type->java opts)) (.setHeaders (:headers opts)) (.setMetricId (parse-metric-id opts)))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Public (schema/defn ^:always-validate request-with-client :- common/ResponsePromise "Issues an async HTTP request with the specified client and returns a promise object to which the value of `(callback {:opts _ :status _ :headers _ :body _})` or `(callback {:opts _ :error _})` will be delivered. When unspecified, `callback` is the identity function. opts: * :url * :method - the HTTP method (:get, :head, :post, :put, :delete, :trace, :options, :patch) * :headers - a map of headers * :body - the body; may be a String or any type supported by clojure's reader * :decompress-body - if `true`, an 'accept-encoding' header with a value of 'gzip, deflate' will be added to the request, and the response will be automatically decompressed if it contains a recognized 'content-encoding' header. defaults to `true`. * :as - used to control the data type of the response body. Supported values are `:text`, `:stream` and `:unbuffered-stream`. `:text` will return a `String`, `:stream` and `:unbuffered-stream` will return an `InputStream`. Note that `:stream` holds the full response in memory (i.e. a `ByteArrayInputStream`). Use `:unbufferred-stream` for large response bodies or to consume less memory. Defaults to `:stream`. * :query-params - used to set the query parameters of an http request * :metric-id - array of strings or keywords, used to set the metrics to be timed for the request." ([opts :- common/RawUserRequestOptions callback :- common/ResponseCallbackFn client :- HttpAsyncClient] (request-with-client opts callback client nil nil true)) ([opts :- common/RawUserRequestOptions callback :- common/ResponseCallbackFn client :- HttpAsyncClient metric-registry :- (schema/maybe MetricRegistry) metric-namespace :- (schema/maybe schema/Str)] (request-with-client opts callback client metric-registry metric-namespace true)) ([opts :- common/RawUserRequestOptions callback :- common/ResponseCallbackFn client :- HttpAsyncClient metric-registry :- (schema/maybe MetricRegistry) metric-namespace :- (schema/maybe schema/Str) enable-url-metrics? :- schema/Bool] (let [result (promise) defaults {:body nil :decompress-body true :compress-request-body :none :as :stream} ^Locale locale (i18n/user-locale) ;; lower-case the header names so that we don't end up with ;; Accept-Language *AND* accept-language in the headers headers (into {"accept-language" (.toLanguageTag locale)} (for [[header value] (:headers opts)] [(str/lower-case header) value])) opts (-> (merge defaults opts) (assoc :headers headers)) java-request-options (clojure-options->java opts) java-method (clojure-method->java opts) response-delivery-delegate (get-response-delivery-delegate opts result)] (JavaClient/requestWithClient java-request-options java-method callback client response-delivery-delegate metric-registry metric-namespace enable-url-metrics?) result))) (schema/defn create-client :- (schema/protocol common/HTTPClient) "Creates a client to be used for making one or more HTTP requests. opts (base set): * :force-redirects - used to set whether or not the client should follow redirects on POST or PUT requests. Defaults to false. * :follow-redirects - used to set whether or not the client should follow redirects in general. Defaults to true. If set to false, will override the :force-redirects setting. * :connect-timeout-milliseconds - maximum number of milliseconds that the client will wait for a connection to be established. A value of zero is interpreted as infinite. A negative value for or the absence of this option is interpreted as undefined (system default). * :socket-timeout-milliseconds - maximum number of milliseconds that the client will allow for no data to be available on the socket before closing the underlying connection, 'SO_TIMEOUT' in socket terms. A timeout of zero is interpreted as an infinite timeout. A negative value for or the absence of this setting is interpreted as undefined (system default). * :ssl-protocols - used to set the list of SSL protocols that the client could select from when talking to the server. Defaults to 'TLSv1', 'TLSv1.1', and 'TLSv1.2'. * :cipher-suites - used to set the cipher suites that the client could select from when talking to the server. Defaults to the complete set of suites supported by the underlying language runtime. * :metric-registry - a MetricRegistry instance used to collect metrics on client requests. opts (ssl-specific where only one of the following combinations permitted): * :ssl-context - an instance of SSLContext OR * :ssl-cert - path to a PEM file containing the client cert * :ssl-key - path to a PEM file containing the client private key * :ssl-ca-cert - path to a PEM file containing the CA cert OR * :ssl-ca-cert - path to a PEM file containing the CA cert" [opts :- common/ClientOptions] (let [client (create-default-client opts) metric-registry (:metric-registry opts) metric-namespace (metrics/build-metric-namespace (:metric-prefix opts) (:server-id opts)) enable-url-metrics? (clojure.core/get opts :enable-url-metrics? true)] (reify common/HTTPClient (get [this url] (common/get this url {})) (get [this url opts] (common/make-request this url :get opts)) (head [this url] (common/head this url {})) (head [this url opts] (common/make-request this url :head opts)) (post [this url] (common/post this url {})) (post [this url opts] (common/make-request this url :post opts)) (put [this url] (common/put this url {})) (put [this url opts] (common/make-request this url :put opts)) (delete [this url] (common/delete this url {})) (delete [this url opts] (common/make-request this url :delete opts)) (trace [this url] (common/trace this url {})) (trace [this url opts] (common/make-request this url :trace opts)) (options [this url] (common/options this url {})) (options [this url opts] (common/make-request this url :options opts)) (patch [this url] (common/patch this url {})) (patch [this url opts] (common/make-request this url :patch opts)) (make-request [this url method] (common/make-request this url method {})) (make-request [_ url method opts] (request-with-client (assoc opts :method method :url url) nil client metric-registry metric-namespace enable-url-metrics?)) (close [_] (.close client)) (get-client-metric-registry [_] metric-registry) (get-client-metric-namespace [_] metric-namespace)))) clj-http-client-0.9.0/src/clj/puppetlabs/http/client/common.clj000066400000000000000000000141071312026620400244550ustar00rootroot00000000000000(ns puppetlabs.http.client.common (:import (java.net URL) (javax.net.ssl SSLContext) (com.codahale.metrics MetricRegistry) (clojure.lang IBlockingDeref) (java.io InputStream) (java.nio.charset Charset) (com.puppetlabs.http.client.metrics ClientTimer)) (:require [schema.core :as schema]) (:refer-clojure :exclude (get))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Client Protocol (defprotocol HTTPClient (get [this url] [this url opts]) (head [this url] [this url opts]) (post [this url] [this url opts]) (put [this url] [this url opts]) (delete [this url] [this url opts]) (trace [this url] [this url opts]) (options [this url] [this url opts]) (patch [this url] [this url opts]) (make-request [this url method] [this url method opts]) (close [this]) (get-client-metric-registry [this]) (get-client-metric-namespace [this])) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Schemas (def ok schema/optional-key) (def UrlOrString (schema/either schema/Str URL)) (def Headers {schema/Str schema/Str}) (def Body (schema/maybe (schema/either String InputStream))) (def BodyType (schema/enum :text :stream :unbuffered-stream)) (def CompressType (schema/enum :gzip :none)) (def MetricId [(schema/either schema/Str schema/Keyword)]) (def RawUserRequestClientOptions "The list of request and client options passed by a user into the request function. Allows the user to configure both a client and a request." {:url UrlOrString :method schema/Keyword (ok :headers) Headers (ok :body) Body (ok :decompress-body) schema/Bool (ok :compress-request-body) CompressType (ok :as) BodyType (ok :query-params) {schema/Str schema/Str} (ok :metric-id) [schema/Str] (ok :ssl-context) SSLContext (ok :ssl-cert) UrlOrString (ok :ssl-key) UrlOrString (ok :ssl-ca-cert) UrlOrString (ok :ssl-protocols) [schema/Str] (ok :cipher-suites) [schema/Str] (ok :force-redirects) schema/Bool (ok :follow-redirects) schema/Bool (ok :connect-timeout-milliseconds) schema/Int (ok :socket-timeout-milliseconds) schema/Int}) (def RawUserRequestOptions "The list of request options passed by a user into the request function. Allows the user to configure a request." {:url UrlOrString :method schema/Keyword (ok :headers) Headers (ok :body) Body (ok :decompress-body) schema/Bool (ok :compress-request-body) CompressType (ok :as) BodyType (ok :query-params) {schema/Str schema/Str} (ok :metric-id) MetricId}) (def RequestOptions "The options from UserRequestOptions that have to do with the configuration and settings for an individual request. This is everything from UserRequestOptions not included in ClientOptions." {:url UrlOrString :method schema/Keyword :headers Headers :body Body :decompress-body schema/Bool :compress-request-body CompressType :as BodyType (ok :query-params) {schema/Str schema/Str} (ok :metric-id) MetricId}) (def SslContextOptions {:ssl-context SSLContext}) (def SslCaCertOptions {:ssl-ca-cert UrlOrString}) (def SslCertOptions {:ssl-cert UrlOrString :ssl-key UrlOrString :ssl-ca-cert UrlOrString}) (def SslProtocolOptions {(ok :ssl-protocols) [schema/Str] (ok :cipher-suites) [schema/Str]}) (def BaseClientOptions {(ok :force-redirects) schema/Bool (ok :follow-redirects) schema/Bool (ok :connect-timeout-milliseconds) schema/Int (ok :socket-timeout-milliseconds) schema/Int (ok :metric-registry) MetricRegistry (ok :server-id) schema/Str (ok :metric-prefix) schema/Str (ok :enable-url-metrics?) schema/Bool}) (def UserRequestOptions "A cleaned-up version of RawUserRequestClientOptions, which is formed after validating the RawUserRequestClientOptions and merging it with the defaults." (schema/either (merge RequestOptions BaseClientOptions) (merge RequestOptions SslContextOptions SslProtocolOptions BaseClientOptions) (merge RequestOptions SslCaCertOptions SslProtocolOptions BaseClientOptions) (merge RequestOptions SslCertOptions SslProtocolOptions BaseClientOptions))) (def ClientOptions "The options from UserRequestOptions that are related to the instantiation/management of a client. This is everything from UserRequestOptions not included in RequestOptions." (schema/either BaseClientOptions (merge SslContextOptions SslProtocolOptions BaseClientOptions) (merge SslCertOptions SslProtocolOptions BaseClientOptions) (merge SslCaCertOptions SslProtocolOptions BaseClientOptions))) (def ResponseCallbackFn (schema/maybe (schema/pred ifn?))) (def ResponsePromise IBlockingDeref) (def ContentType (schema/maybe {:mime-type schema/Str :charset (schema/maybe Charset)})) (def NormalResponse {:opts UserRequestOptions :orig-content-encoding (schema/maybe schema/Str) :body Body :headers Headers :status schema/Int :content-type ContentType}) (def ErrorResponse {:opts UserRequestOptions :error Exception}) (def Response (schema/either NormalResponse ErrorResponse)) (def HTTPMethod (schema/enum :delete :get :head :option :patch :post :put :trace)) (def Metrics [ClientTimer]) (def AllMetrics {:url Metrics :url-and-method Metrics :metric-id Metrics}) (def BaseMetricData {:metric-name schema/Str :count schema/Int :mean schema/Num :aggregate schema/Num}) (def UrlMetricData (assoc BaseMetricData :url schema/Str)) (def UrlAndMethodMetricData (assoc UrlMetricData :method schema/Str)) (def MetricIdMetricData (assoc BaseMetricData :metric-id [schema/Str])) (def AllMetricsData {:url [UrlMetricData] :url-and-method [UrlAndMethodMetricData] :metric-id [MetricIdMetricData]}) (def MetricTypes (schema/enum :full-response)) clj-http-client-0.9.0/src/clj/puppetlabs/http/client/metrics.clj000066400000000000000000000111161312026620400246300ustar00rootroot00000000000000(ns puppetlabs.http.client.metrics (:require [schema.core :as schema] [puppetlabs.http.client.common :as common]) (:import (com.puppetlabs.http.client.metrics Metrics$MetricType Metrics ClientMetricData) (com.codahale.metrics MetricRegistry))) (schema/defn get-base-metric-data :- common/BaseMetricData [data :- ClientMetricData] {:count (.getCount data) :mean (.getMean data) :aggregate (.getAggregate data) :metric-name (.getMetricName data)}) (schema/defn get-url-metric-data :- common/UrlMetricData [data :- ClientMetricData] (assoc (get-base-metric-data data) :url (.getUrl data))) (schema/defn get-url-and-method-metric-data :- common/UrlAndMethodMetricData [data :- ClientMetricData] (assoc (get-url-metric-data data) :method (.getMethod data))) (schema/defn get-metric-id-metric-data :- common/MetricIdMetricData [data :- ClientMetricData] (assoc (get-base-metric-data data) :metric-id (.getMetricId data))) (defn get-java-metric-type [metric-type] (case metric-type :full-response Metrics$MetricType/FULL_RESPONSE)) (defn uppercase-method [method] (clojure.string/upper-case (name method))) (defn build-metric-namespace [metric-prefix server-id] (Metrics/buildMetricNamespace metric-prefix server-id)) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Public (schema/defn ^:always-validate url->metric-url :- schema/Str [url :- schema/Str] (Metrics/urlToMetricUrl url)) (schema/defn ^:always-validate get-client-metrics :- (schema/maybe common/AllMetrics) "Returns the http client-specific metrics from the metric registry." [metric-registry :- MetricRegistry] (let [metrics (Metrics/getClientMetrics metric-registry)] {:url (.getUrlTimers metrics) :url-and-method (.getUrlAndMethodTimers metrics) :metric-id (.getMetricIdTimers metrics)})) (schema/defn ^:always-validate get-client-metrics-by-url :- common/Metrics "Returns the http client-specific url metrics matching the specified url." [metric-registry :- MetricRegistry url :- schema/Str] (Metrics/getClientMetricsByUrl metric-registry url)) (schema/defn ^:always-validate get-client-metrics-by-url-and-method :- common/Metrics "Returns the http client-specific url metrics matching the specified url." [metric-registry :- MetricRegistry url :- schema/Str method :- common/HTTPMethod] (Metrics/getClientMetricsByUrlAndMethod metric-registry url method)) (schema/defn ^:always-validate get-client-metrics-by-metric-id :- common/Metrics "Returns the http client-specific url metrics matching the specified url." [metric-registry :- MetricRegistry metric-id :- common/MetricId] (Metrics/getClientMetricsByMetricId metric-registry (into-array String (map name metric-id)))) (schema/defn ^:always-validate get-client-metrics-data :- common/AllMetricsData "Returns a summary of the metric data for all http client timers, organized in a map by category." [metric-registry :- MetricRegistry] (let [data (Metrics/getClientMetricsData metric-registry)] {:url (map get-url-metric-data (.getUrlData data)) :url-and-method (map get-url-and-method-metric-data (.getUrlAndMethodData data)) :metric-id (map get-metric-id-metric-data (.getMetricIdData data))})) (schema/defn ^:always-validate get-client-metrics-data-by-url :- [common/UrlMetricData] "Returns a summary of the metric data for all http client timers filtered by url." [metric-registry :- MetricRegistry url :- schema/Str] (let [data (Metrics/getClientMetricsDataByUrl metric-registry url)] (map get-url-metric-data data))) (schema/defn ^:always-validate get-client-metrics-data-by-url-and-method :- [common/UrlAndMethodMetricData] "Returns a summary of the metric data for all http client timers filtered by url and method." [metric-registry :- MetricRegistry url :- schema/Str method :- common/HTTPMethod] (let [data (Metrics/getClientMetricsDataByUrlAndMethod metric-registry url (uppercase-method method))] (map get-url-and-method-metric-data data))) (schema/defn ^:always-validate get-client-metrics-data-by-metric-id :- [common/MetricIdMetricData] "Returns a summary of the metric data for all http client timers filtered by metric-id." [metric-registry :- MetricRegistry metric-id :- common/MetricId] (let [data (Metrics/getClientMetricsDataByMetricId metric-registry (into-array String (map name metric-id)))] (map get-metric-id-metric-data data))) clj-http-client-0.9.0/src/clj/puppetlabs/http/client/sync.clj000066400000000000000000000123341312026620400241410ustar00rootroot00000000000000;; This namespace provides synchronous versions of the request functions ;; defined in puppetlabs.http.client (ns puppetlabs.http.client.sync (:require [puppetlabs.http.client.async :as async] [schema.core :as schema] [puppetlabs.http.client.common :as common] [puppetlabs.http.client.metrics :as metrics]) (:refer-clojure :exclude (get))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Private utility functions (schema/defn extract-client-opts :- common/ClientOptions [opts :- common/RawUserRequestClientOptions] (select-keys opts [:ssl-context :ssl-ca-cert :ssl-cert :ssl-key :ssl-protocols :cipher-suites :force-redirects :follow-redirects :connect-timeout-milliseconds :socket-timeout-milliseconds])) (schema/defn extract-request-opts :- common/RawUserRequestOptions [opts :- common/RawUserRequestClientOptions] (select-keys opts [:url :method :headers :body :decompress-body :compress-request-body :as :query-params])) (defn request-with-client ([req client] (request-with-client req client nil nil true)) ([req client metric-registry metric-namespace] (request-with-client req client metric-registry metric-namespace true)) ([req client metric-registry metric-namespace enable-url-metrics?] (let [{:keys [error] :as resp} @(async/request-with-client req nil client metric-registry metric-namespace enable-url-metrics?)] (if error (throw error) resp)))) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;; Public (defn request [req] (with-open [client (async/create-default-client (extract-client-opts req))] (request-with-client (extract-request-opts req) client))) (schema/defn create-client :- (schema/protocol common/HTTPClient) [opts :- common/ClientOptions] (let [client (async/create-default-client opts) metric-registry (:metric-registry opts) metric-namespace (metrics/build-metric-namespace (:metric-prefix opts) (:server-id opts)) enable-url-metrics? (clojure.core/get opts :enable-url-metrics? true)] (reify common/HTTPClient (get [this url] (common/get this url {})) (get [this url opts] (common/make-request this url :get opts)) (head [this url] (common/head this url {})) (head [this url opts] (common/make-request this url :head opts)) (post [this url] (common/post this url {})) (post [this url opts] (common/make-request this url :post opts)) (put [this url] (common/put this url {})) (put [this url opts] (common/make-request this url :put opts)) (delete [this url] (common/delete this url {})) (delete [this url opts] (common/make-request this url :delete opts)) (trace [this url] (common/trace this url {})) (trace [this url opts] (common/make-request this url :trace opts)) (options [this url] (common/options this url {})) (options [this url opts] (common/make-request this url :post opts)) (patch [this url] (common/patch this url {})) (patch [this url opts] (common/make-request this url :patch opts)) (make-request [this url method] (common/make-request this url method {})) (make-request [_ url method opts] (request-with-client (assoc opts :method method :url url) client metric-registry metric-namespace enable-url-metrics?)) (close [_] (.close client)) (get-client-metric-registry [_] metric-registry) (get-client-metric-namespace [_] metric-namespace)))) (defn get "Issue a synchronous HTTP GET request. This will raise an exception if an error is returned." ([url] (get url {})) ([url opts] (request (assoc opts :method :get :url url)))) (defn head "Issue a synchronous HTTP head request. This will raise an exception if an error is returned." ([url] (head url {})) ([url opts] (request (assoc opts :method :head :url url)))) (defn post "Issue a synchronous HTTP POST request. This will raise an exception if an error is returned." ([url] (post url {})) ([url opts] (request (assoc opts :method :post :url url)))) (defn put "Issue a synchronous HTTP PUT request. This will raise an exception if an error is returned." ([url] (put url {})) ([url opts] (request (assoc opts :method :put :url url)))) (defn delete "Issue a synchronous HTTP DELETE request. This will raise an exception if an error is returned." ([url] (delete url {})) ([url opts] (request (assoc opts :method :delete :url url)))) (defn trace "Issue a synchronous HTTP TRACE request. This will raise an exception if an error is returned." ([url] (trace url {})) ([url opts] (request (assoc opts :method :trace :url url)))) (defn options "Issue a synchronous HTTP OPTIONS request. This will raise an exception if an error is returned." ([url] (options url {})) ([url opts] (request (assoc opts :method :options :url url)))) (defn patch "Issue a synchronous HTTP PATCH request. This will raise an exception if an error is returned." ([url] (patch url {})) ([url opts] (request (assoc opts :method :patch :url url)))) clj-http-client-0.9.0/src/java/000077500000000000000000000000001312026620400162255ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/000077500000000000000000000000001312026620400170035ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/000077500000000000000000000000001312026620400211625ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/000077500000000000000000000000001312026620400221415ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/000077500000000000000000000000001312026620400234175ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/Async.java000066400000000000000000000021431312026620400253370ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.puppetlabs.http.client.impl.JavaClient; import com.puppetlabs.http.client.impl.PersistentAsyncHttpClient; import com.puppetlabs.http.client.metrics.Metrics; /** * This class allows you to create an AsyncHttpClient for use in making * HTTP Requests. It consists exclusively of a static method to create * a client. */ public class Async { /** * Allows you to create an instance of an AsyncHttpClient for use in * making HTTP requests. * * @param clientOptions the list of options with which to configure the client * @return an AsyncHttpClient that can be used to make requests */ public static AsyncHttpClient createClient(ClientOptions clientOptions) { final String metricNamespace = Metrics.buildMetricNamespace(clientOptions.getMetricPrefix(), clientOptions.getServerId()); return new PersistentAsyncHttpClient(JavaClient.createClient(clientOptions), clientOptions.getMetricRegistry(), metricNamespace, clientOptions.isEnableURLMetrics()); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/AsyncHttpClient.java000066400000000000000000000150211312026620400273350ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.codahale.metrics.MetricRegistry; import com.puppetlabs.http.client.impl.Promise; import java.io.Closeable; import java.net.URI; import java.net.URISyntaxException; /** * This interface represents an asynchronous HTTP client with which * requests can be made. An object implementing this interface is returned by * {@link com.puppetlabs.http.client.Async#createClient(ClientOptions)}. */ public interface AsyncHttpClient extends Closeable{ /** * @return the MetricRegistry instance associated with this Client */ public MetricRegistry getMetricRegistry(); /** * @return the String metricNamespace for this Client */ public String getMetricNamespace(); /** * Performs a GET request * @param url the URL against which to make the GET request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise get(String url) throws URISyntaxException; /** * Performs a GET request * @param uri the URI against which to make the GET request * @return a Promise with the contents of the response */ public Promise get(URI uri); /** * Performs a GET request * @param requestOptions options to configure the GET request * @return a Promise with the contents of the response */ public Promise get(RequestOptions requestOptions); /** * Performs a HEAD request * @param url the URL against which to make the HEAD request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise head(String url) throws URISyntaxException; /** * Performs a HEAD request * @param uri the URI against which to make the HEAD request * @return a Promise with the contents of the response */ public Promise head(URI uri); /** * Performs a HEAD request * @param requestOptions options to configure the HEAD request * @return a Promise with the contents of the response */ public Promise head(RequestOptions requestOptions); /** * Performs a POST request * @param url the URL against which to make the POST request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise post(String url) throws URISyntaxException; /** * Performs a POST request * @param uri the URI against which to make the POST request * @return a Promise with the contents of the response */ public Promise post(URI uri); /** * Performs a POST request * @param requestOptions options to configure the POST request * @return a Promise with the contents of the response */ public Promise post(RequestOptions requestOptions); /** * Performs a PUT request * @param url the URL against which to make the PUT request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise put(String url) throws URISyntaxException; /** * Performs a PUT request * @param uri the URI against which to make the PUT request * @return a Promise with the contents of the response */ public Promise put(URI uri); /** * Performs a PUT request * @param requestOptions options to configure the PUT request * @return a Promise with the contents of the response */ public Promise put(RequestOptions requestOptions); /** * Performs a DELETE request * @param url the URL against which to make the DELETE request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise delete(String url) throws URISyntaxException; /** * Performs a DELETE request * @param uri the URI against which to make the DELETE request * @return a Promise with the contents of the response */ public Promise delete(URI uri); /** * Performs a DELETE request * @param requestOptions options to configure the DELETE request * @return a Promise with the contents of the response */ public Promise delete(RequestOptions requestOptions); /** * Performs a TRACE request * @param url the URL against which to make the TRACE request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise trace(String url) throws URISyntaxException; /** * Performs a TRACE request * @param uri the URI against which to make the TRACE request * @return a Promise with the contents of the response */ public Promise trace(URI uri); /** * Performs a TRACE request * @param requestOptions options to configure the TRACE request * @return a Promise with the contents of the response */ public Promise trace(RequestOptions requestOptions); /** * Performs an OPTIONS request * @param url the URL against which to make the OPTIONS request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise options(String url) throws URISyntaxException; /** * Performs an OPTIONS request * @param uri the URI against which to make the OPTIONS request * @return a Promise with the contents of the response */ public Promise options(URI uri); /** * Performs an OPTIONS request * @param requestOptions options to configure the OPTIONS request * @return a Promise with the contents of the response */ public Promise options(RequestOptions requestOptions); /** * Performs a PATCH request * @param url the URL against which to make the PATCH request * @return a Promise with the contents of the response * @throws URISyntaxException */ public Promise patch(String url) throws URISyntaxException; /** * Performs a PATCH request * @param uri the URI against which to make the PATCH request * @return a Promise with the contents of the response */ public Promise patch(URI uri); /** * Performs a PATCH request * @param requestOptions options to configure the PATCH request * @return a Promise with the contents of the response */ public Promise patch(RequestOptions requestOptions); } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/ClientOptions.java000066400000000000000000000166611312026620400270660ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.codahale.metrics.MetricRegistry; import javax.net.ssl.SSLContext; /** * This class is a wrapper around a number of options for use * in configuring a persistent HTTP Client. * * @see com.puppetlabs.http.client.Async#createClient(ClientOptions) * @see com.puppetlabs.http.client.Sync#createClient(ClientOptions) */ public class ClientOptions { public static final String[] DEFAULT_SSL_PROTOCOLS = new String[] {"TLSv1", "TLSv1.1", "TLSv1.2"}; private SSLContext sslContext; private String sslCert; private String sslKey; private String sslCaCert; private String[] sslProtocols; private String[] sslCipherSuites; private boolean insecure = false; private boolean forceRedirects = false; private boolean followRedirects = true; private int connectTimeoutMilliseconds = -1; private int socketTimeoutMilliseconds = -1; private MetricRegistry metricRegistry; private String metricPrefix; private String serverId; private boolean enableURLMetrics = true; /** * Constructor for the ClientOptions class. When this constructor is called, * insecure and forceRedirects will default to false, and followRedirects will default * to true. */ public ClientOptions() { } /** * Constructor for the ClientOptions class. * @param sslContext The SSL context for the client. * @param sslCert The path to a PEM file containing the client cert * @param sslKey The path to a PEM file containing the client private key * @param sslCaCert The path to a PEM file containing the CA cert * @param sslProtocols The SSL protocols that the client can select from when talking to the server * @param sslCipherSuites The cipher suites that the client can select from when talking to the server * @param insecure Whether or not the client should use an insecure connection. * @param forceRedirects Whether or not the client should follow redirects on POST or PUT requests. * @param followRedirects Whether or not the client should follow redirects in general. * @param connectTimeoutMilliseconds Maximum number of milliseconds that * the client will wait for a * connection to be established. A * value of zero is interpreted as * infinite. A negative value is * interpreted as undefined (system * default). * @param socketTimeoutMilliseconds Maximum number of milliseconds that * the client will allow for no data to * be available on the socket before * closing the underlying connection, * SO_TIMEOUT in socket * terms. A timeout of zero is * interpreted as an infinite timeout. * A negative value is interpreted as * undefined (system default). */ public ClientOptions(SSLContext sslContext, String sslCert, String sslKey, String sslCaCert, String[] sslProtocols, String[] sslCipherSuites, boolean insecure, boolean forceRedirects, boolean followRedirects, int connectTimeoutMilliseconds, int socketTimeoutMilliseconds) { this.sslContext = sslContext; this.sslCert = sslCert; this.sslKey = sslKey; this.sslCaCert = sslCaCert; this.sslProtocols = sslProtocols; this.sslCipherSuites = sslCipherSuites; this.insecure = insecure; this.forceRedirects = forceRedirects; this.followRedirects = followRedirects; this.connectTimeoutMilliseconds = connectTimeoutMilliseconds; this.socketTimeoutMilliseconds = socketTimeoutMilliseconds; } public SSLContext getSslContext() { return sslContext; } public ClientOptions setSslContext(SSLContext sslContext) { this.sslContext = sslContext; return this; } public String getSslCert() { return sslCert; } public ClientOptions setSslCert(String sslCert) { this.sslCert = sslCert; return this; } public String getSslKey() { return sslKey; } public ClientOptions setSslKey(String sslKey) { this.sslKey = sslKey; return this; } public String getSslCaCert() { return sslCaCert; } public ClientOptions setSslCaCert(String sslCaCert) { this.sslCaCert = sslCaCert; return this; } public String[] getSslProtocols() { return sslProtocols; } public ClientOptions setSslProtocols(String[] sslProtocols) { this.sslProtocols = sslProtocols; return this; } public String[] getSslCipherSuites() { return sslCipherSuites; } public ClientOptions setSslCipherSuites(String[] sslCipherSuites) { this.sslCipherSuites = sslCipherSuites; return this; } public boolean getInsecure() { return insecure; } public ClientOptions setInsecure(boolean insecure) { this.insecure = insecure; return this; } public boolean getForceRedirects() { return forceRedirects; } public ClientOptions setForceRedirects(boolean forceRedirects) { this.forceRedirects = forceRedirects; return this; } public boolean getFollowRedirects() { return followRedirects; } public ClientOptions setFollowRedirects(boolean followRedirects) { this.followRedirects = followRedirects; return this; } public int getConnectTimeoutMilliseconds() { return connectTimeoutMilliseconds; } public ClientOptions setConnectTimeoutMilliseconds( int connectTimeoutMilliseconds) { this.connectTimeoutMilliseconds = connectTimeoutMilliseconds; return this; } public int getSocketTimeoutMilliseconds() { return socketTimeoutMilliseconds; } public ClientOptions setSocketTimeoutMilliseconds( int socketTimeoutMilliseconds) { this.socketTimeoutMilliseconds = socketTimeoutMilliseconds; return this; } public MetricRegistry getMetricRegistry() { return metricRegistry; } public ClientOptions setMetricRegistry(MetricRegistry metricRegistry) { this.metricRegistry = metricRegistry; return this; } public String getMetricPrefix() { return metricPrefix; } public ClientOptions setMetricPrefix(String metricPrefix) { this.metricPrefix = metricPrefix; return this; } public String getServerId() { return serverId; } public ClientOptions setServerId(String serverId) { this.serverId = serverId; return this; } public boolean isEnableURLMetrics() { return enableURLMetrics; } public ClientOptions setEnableURLMetrics(boolean enableURLMetrics) { this.enableURLMetrics = enableURLMetrics; return this; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/CompressType.java000066400000000000000000000001251312026620400267150ustar00rootroot00000000000000package com.puppetlabs.http.client; public enum CompressType { GZIP, NONE } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/HttpClientException.java000066400000000000000000000005301312026620400302150ustar00rootroot00000000000000package com.puppetlabs.http.client; /** * This class represents an exception caused by an error in * an HTTP request */ public class HttpClientException extends RuntimeException { public HttpClientException(String msg) { super(msg); } public HttpClientException(String msg, Throwable t) { super(msg, t); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/HttpMethod.java000066400000000000000000000012461312026620400263450ustar00rootroot00000000000000package com.puppetlabs.http.client; import org.apache.http.client.methods.*; /** * This enum represents the various HTTP methods that can be used to make requests. */ public enum HttpMethod { GET(HttpGet.class), HEAD(HttpHead.class), POST(HttpPost.class), PUT(HttpPut.class), DELETE(HttpDelete.class), TRACE(HttpTrace.class), OPTIONS(HttpOptions.class), PATCH(HttpPatch.class); private Class httpMethod; HttpMethod(Class httpMethod) { this.httpMethod = httpMethod; } public Class getValue() { return this.httpMethod; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/RequestOptions.java000066400000000000000000000066721312026620400273010ustar00rootroot00000000000000package com.puppetlabs.http.client; import java.net.URI; import java.net.URISyntaxException; import java.util.Map; /** * This class is a wrapper around a number of options for use in * configuring an HTTP request. */ public class RequestOptions { private URI uri; private Map headers; private Object body; private CompressType requestBodyCompression = CompressType.NONE; private boolean decompressBody = true; private ResponseBodyType as = ResponseBodyType.STREAM; private String[] metricId; /** * Constructor for the RequestOptions class. When this constructor is called, * decompressBody will default to true, and as will default to ResponseBodyType.STREAM * @param url the URL against which to make the request * @throws URISyntaxException */ public RequestOptions (String url) throws URISyntaxException { this.uri = new URI(url); } /** * Constructor for the RequestOptions class. When this constructor is called, * decompressBody will default to true, and as will default to ResponseBodyType.STREAM * @param uri the URI against which to make the request */ public RequestOptions(URI uri) { this.uri = uri; } /** * Constructor for the RequestOptions class * @param uri the URI against which to make the request * @param headers A map of headers. Can be null. * @param body The body of the request. Can be null. * @param decompressBody If true, an "accept-encoding" header with a value of "gzip, deflate" will be * automatically decompressed if it contains a recognized "content-encoding" header. * @param as Used to control the data type of the response body. */ public RequestOptions (URI uri, Map headers, Object body, boolean decompressBody, ResponseBodyType as) { this.uri = uri; this.headers = headers; this.body = body; this.decompressBody = decompressBody; this.as = as; } public URI getUri() { return uri; } public RequestOptions setUri(URI uri) { this.uri = uri; return this; } public Map getHeaders() { return headers; } public RequestOptions setHeaders(Map headers) { this.headers = headers; return this; } public Object getBody() { return body; } public RequestOptions setBody(Object body) { this.body = body; return this; } public boolean getDecompressBody() { return decompressBody; } public RequestOptions setDecompressBody(boolean decompressBody) { this.decompressBody = decompressBody; return this; } public CompressType getCompressRequestBody() { return requestBodyCompression; } public RequestOptions setCompressRequestBody( CompressType requestBodyCompression) { this.requestBodyCompression = requestBodyCompression; return this; } public ResponseBodyType getAs() { return as; } public RequestOptions setAs(ResponseBodyType as) { this.as = as; return this; } public String[] getMetricId() { return metricId; } public RequestOptions setMetricId(String[] metricId) { this.metricId = metricId; return this; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/Response.java000066400000000000000000000027371312026620400260710ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.puppetlabs.http.client.RequestOptions; import org.apache.http.entity.ContentType; import java.util.Map; /** * This class represents a response from an HTTP request. */ public class Response { private RequestOptions options; private String origContentEncoding; private Throwable error; private Object body; private Map headers; private Integer status; private ContentType contentType; public Response(RequestOptions options, Throwable error) { this.options = options; this.error = error; } public Response(RequestOptions options, String origContentEncoding, Object body, Map headers, int status, ContentType contentType) { this.options = options; this.origContentEncoding = origContentEncoding; this.body = body; this.headers = headers; this.status = status; this.contentType = contentType; } public RequestOptions getOptions() { return options; } public String getOrigContentEncoding() { return origContentEncoding; } public Throwable getError() { return error; } public Object getBody() { return body; } public Map getHeaders() { return headers; } public Integer getStatus() { return status; } public ContentType getContentType() { return contentType; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/ResponseBodyType.java000066400000000000000000000003011312026620400275320ustar00rootroot00000000000000package com.puppetlabs.http.client; /** * This Enum represents the possible types of the body of a response. */ public enum ResponseBodyType { UNBUFFERED_STREAM, STREAM, TEXT; } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/SimpleRequestOptions.java000066400000000000000000000134231312026620400304430ustar00rootroot00000000000000package com.puppetlabs.http.client; import javax.net.ssl.SSLContext; import java.net.URI; import java.net.URISyntaxException; import java.util.Map; /** * This class represents is a wrapper around a number of options for use in * making requests with the simple request functions contained in the Sync class. * It is a combination of the options from ClientOptions and RequestOptions. * * @see com.puppetlabs.http.client.ClientOptions#ClientOptions(javax.net.ssl.SSLContext, String, String, String, String[], String[], boolean, boolean, boolean) * @see com.puppetlabs.http.client.RequestOptions#RequestOptions(java.net.URI, java.util.Map, Object, boolean, ResponseBodyType) */ public class SimpleRequestOptions { private URI uri; private Map headers; private SSLContext sslContext; private String sslCert; private String sslKey; private String sslCaCert; private String[] sslProtocols; private String[] sslCipherSuites; private boolean insecure = false; private Object body; private CompressType requestBodyCompression = CompressType.NONE; private boolean decompressBody = true; private ResponseBodyType as = ResponseBodyType.STREAM; private boolean forceRedirects = false; private boolean followRedirects = true; private int connectTimeoutMilliseconds = -1; private int socketTimeoutMilliseconds = -1; /** * Constructor for SimpleRequestOptions. When this constructor is used, * insecure and forceRedirects default to false, and followRedirects and decompressBody * default to true. as defaults to ResponseBodyType.STREAM. * @param url the URL against which to make the HTTP request * @throws URISyntaxException */ public SimpleRequestOptions (String url) throws URISyntaxException { this.uri = new URI(url); } /** * Constructor for SimpleRequestOptions. When this constructor is used, * insecure and forceRedirects default to false, and followRedirects and decompressBody * default to true. as defaults to ResponseBodyType.STREAM. * @param uri the URI against which to make the HTTP request */ public SimpleRequestOptions(URI uri) { this.uri = uri; } public URI getUri() { return uri; } public SimpleRequestOptions setUri(URI uri) { this.uri = uri; return this; } public Map getHeaders() { return headers; } public SimpleRequestOptions setHeaders(Map headers) { this.headers = headers; return this; } public SSLContext getSslContext() { return sslContext; } public SimpleRequestOptions setSslContext(SSLContext sslContext) { this.sslContext = sslContext; return this; } public String getSslCert() { return sslCert; } public SimpleRequestOptions setSslCert(String sslCert) { this.sslCert = sslCert; return this; } public String getSslKey() { return sslKey; } public SimpleRequestOptions setSslKey(String sslKey) { this.sslKey = sslKey; return this; } public String getSslCaCert() { return sslCaCert; } public SimpleRequestOptions setSslCaCert(String sslCaCert) { this.sslCaCert = sslCaCert; return this; } public String[] getSslProtocols() { return sslProtocols; } public SimpleRequestOptions setSslProtocols(String[] sslProtocols) { this.sslProtocols = sslProtocols; return this; } public String[] getSslCipherSuites() { return sslCipherSuites; } public SimpleRequestOptions setSslCipherSuites(String[] sslCipherSuites) { this.sslCipherSuites = sslCipherSuites; return this; } public boolean getInsecure() { return insecure; } public SimpleRequestOptions setInsecure(boolean insecure) { this.insecure = insecure; return this; } public Object getBody() { return body; } public SimpleRequestOptions setBody(Object body) { this.body = body; return this; } public boolean getDecompressBody() { return decompressBody; } public SimpleRequestOptions setDecompressBody(boolean decompressBody) { this.decompressBody = decompressBody; return this; } public CompressType getCompressRequestBody() { return requestBodyCompression; } public SimpleRequestOptions setRequestBodyCompression( CompressType requestBodyCompression) { this.requestBodyCompression = requestBodyCompression; return this; } public ResponseBodyType getAs() { return as; } public SimpleRequestOptions setAs(ResponseBodyType as) { this.as = as; return this; } public boolean getForceRedirects() { return forceRedirects; } public SimpleRequestOptions setForceRedirects(boolean forceRedirects) { this.forceRedirects = forceRedirects; return this; } public boolean getFollowRedirects() { return followRedirects; } public SimpleRequestOptions setFollowRedirects(boolean followRedirects) { this.followRedirects = followRedirects; return this; } public int getConnectTimeoutMilliseconds() { return connectTimeoutMilliseconds; } public SimpleRequestOptions setConnectTimeoutMilliseconds( int connectTimeoutMilliseconds) { this.connectTimeoutMilliseconds = connectTimeoutMilliseconds; return this; } public int getSocketTimeoutMilliseconds() { return socketTimeoutMilliseconds; } public SimpleRequestOptions setSocketTimeoutMilliseconds( int socketTimeoutMilliseconds) { this.socketTimeoutMilliseconds = socketTimeoutMilliseconds; return this; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/Sync.java000066400000000000000000000270511312026620400252030ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.puppetlabs.http.client.impl.JavaClient; import com.puppetlabs.http.client.impl.PersistentSyncHttpClient; import com.puppetlabs.http.client.metrics.Metrics; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import javax.net.ssl.SSLContext; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; import java.util.Map; /** * This class allows for the creation of a persistent synchronous HTTP client. It also allows * for sending synchronous HTTP requests without a persistent HTTP client. */ public class Sync { private static final Logger LOGGER = LoggerFactory.getLogger(Sync.class); private static void logAndRethrow(String msg, Throwable t) { LOGGER.error(msg, t); throw new HttpClientException(msg, t); } private static RequestOptions extractRequestOptions(SimpleRequestOptions simpleOptions) { URI uri = simpleOptions.getUri(); Map headers = simpleOptions.getHeaders(); Object body = simpleOptions.getBody(); boolean decompressBody = simpleOptions.getDecompressBody(); ResponseBodyType as = simpleOptions.getAs(); CompressType requestBodyDecompression = simpleOptions.getCompressRequestBody(); RequestOptions requestOptions = new RequestOptions( uri, headers, body, decompressBody, as); requestOptions.setCompressRequestBody(requestBodyDecompression); return requestOptions; } private static ClientOptions extractClientOptions(SimpleRequestOptions simpleOptions) { SSLContext sslContext = simpleOptions.getSslContext(); String sslCert = simpleOptions.getSslCert(); String sslKey = simpleOptions.getSslKey(); String sslCaCert = simpleOptions.getSslCaCert(); String[] sslProtocols = simpleOptions.getSslProtocols(); String[] sslCipherSuites = simpleOptions.getSslCipherSuites(); boolean insecure = simpleOptions.getInsecure(); boolean forceRedirects = simpleOptions.getForceRedirects(); boolean followRedirects = simpleOptions.getFollowRedirects(); int connectTimeoutMilliseconds = simpleOptions.getConnectTimeoutMilliseconds(); int socketTimeoutMilliseconds = simpleOptions.getSocketTimeoutMilliseconds(); return new ClientOptions(sslContext, sslCert, sslKey, sslCaCert, sslProtocols, sslCipherSuites, insecure, forceRedirects, followRedirects, connectTimeoutMilliseconds, socketTimeoutMilliseconds); } private static Response request(SimpleRequestOptions simpleRequestOptions, HttpMethod method) { // TODO: if we end up implementing an async version of the java API, // we should refactor this implementation so that it is based on the // async one, as Patrick has done in the clojure API. Response response = null; final SyncHttpClient client = createClient( extractClientOptions(simpleRequestOptions)); try { response = client.request( extractRequestOptions(simpleRequestOptions), method); } finally { try { client.close(); } catch (IOException e) { logAndRethrow("Error closing client", e); } } return response; } /** * Creates a synchronous persistent HTTP client * @param clientOptions * @return A persistent synchronous HTTP client */ public static SyncHttpClient createClient(ClientOptions clientOptions) { final String metricNamespace = Metrics.buildMetricNamespace(clientOptions.getMetricPrefix(), clientOptions.getServerId()); return new PersistentSyncHttpClient(JavaClient.createClient(clientOptions), clientOptions.getMetricRegistry(), metricNamespace, clientOptions.isEnableURLMetrics()); } /** * Makes a simple HTTP GET request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response get(String url) throws URISyntaxException { return get(new URI(url)); } /** * Makes a simple HTTP GET request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response get(URI uri) { return get(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP GET request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response get(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.GET); } /** * Makes a simple HTTP HEAD request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response head(String url) throws URISyntaxException { return head(new URI(url)); } /** * Makes a simple HTTP HEAD request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response head(URI uri) { return head(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP HEAD request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response head(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.HEAD); } /** * Makes a simple HTTP POST request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response post(String url) throws URISyntaxException { return post(new URI(url)); } /** * Makes a simple HTTP POST request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response post(URI uri) { return post(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP POST request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response post(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.POST); } /** * Makes a simple HTTP PUT request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response put(String url) throws URISyntaxException { return put(new URI(url)); } /** * Makes a simple HTTP PUT request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response put(URI uri) { return put(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP PUT request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response put(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.PUT); } /** * Makes a simple HTTP DELETE request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response delete(String url) throws URISyntaxException { return delete(new URI(url)); } /** * Makes a simple HTTP DELETE request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response delete(URI uri) { return delete(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP DELETE request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response delete(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.DELETE); } /** * Makes a simple HTTP TRACE request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response trace(String url) throws URISyntaxException { return trace(new URI(url)); } /** * Makes a simple HTTP TRACE request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response trace(URI uri) { return trace(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP TRACE request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response trace(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.TRACE); } /** * Makes a simple HTTP OPTIONS request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response options(String url) throws URISyntaxException { return options(new URI(url)); } /** * Makes a simple HTTP OPTIONS request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response options(URI uri) { return options(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP OPTIONS request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response options(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.OPTIONS); } /** * Makes a simple HTTP PATCH request * @param url The URL against which to make the request * @return The HTTP Response corresponding to the request * @throws URISyntaxException */ public static Response patch(String url) throws URISyntaxException { return patch(new URI(url)); } /** * Makes a simple HTTP PATCH request * @param uri The URI against which to make the request * @return The HTTP Response corresponding to the request */ public static Response patch(URI uri) { return patch(new SimpleRequestOptions(uri)); } /** * Makes a simple HTTP PATCH request * @param simpleRequestOptions The options to configure the request and the client making it * @return The HTTP response corresponding to the request */ public static Response patch(SimpleRequestOptions simpleRequestOptions) { return request(simpleRequestOptions, HttpMethod.PATCH); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/SyncHttpClient.java000066400000000000000000000136211312026620400272000ustar00rootroot00000000000000package com.puppetlabs.http.client; import com.codahale.metrics.MetricRegistry; import java.io.Closeable; import java.net.URI; import java.net.URISyntaxException; /** * This interface represents a synchronous HTTP client with which * requests can be made. An object implementing this interface is * returned by {@link com.puppetlabs.http.client.Sync#createClient(ClientOptions)} */ public interface SyncHttpClient extends Closeable { /** * @return the MetricRegistry instance associated with this Client */ public MetricRegistry getMetricRegistry(); /** * @return the String metricNamespace for this Client */ public String getMetricNamespace(); /** * Makes a configurable HTTP request * @param requestOptions the options to configure the request * @param method the type of the HTTP request * @return the HTTP response */ public Response request(RequestOptions requestOptions, HttpMethod method); /** * Makes an HTTP GET request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response get(String url) throws URISyntaxException; /** * Makes an HTTP GET request * @param uri the uri against which to make the request * @return the HTTP response */ public Response get(URI uri); /** * Makes an HTTP GET request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response get(RequestOptions requestOptions); /** * Makes an HTTP HEAD request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response head(String url) throws URISyntaxException; /** * Makes an HTTP HEAD request * @param uri the uri against which to make the request * @return the HTTP response */ public Response head(URI uri); /** * Makes an HTTP HEAD request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response head(RequestOptions requestOptions); /** * Makes an HTTP POST request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response post(String url) throws URISyntaxException; /** * Makes an HTTP POST request * @param uri the uri against which to make the request * @return the HTTP response */ public Response post(URI uri); /** * Makes an HTTP POST request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response post(RequestOptions requestOptions); /** * Makes an HTTP PUT request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response put(String url) throws URISyntaxException; /** * Makes an HTTP PUT request * @param uri the uri against which to make the request * @return the HTTP response */ public Response put(URI uri); /** * Makes an HTTP PUT request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response put(RequestOptions requestOptions); /** * Makes an HTTP DELETE request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response delete(String url) throws URISyntaxException; /** * Makes an HTTP DELETE request * @param uri the uri against which to make the request * @return the HTTP response */ public Response delete(URI uri); /** * Makes an HTTP DELETE request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response delete(RequestOptions requestOptions); /** * Makes an HTTP TRACE request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response trace(String url) throws URISyntaxException; /** * Makes an HTTP TRACE request * @param uri the uri against which to make the request * @return the HTTP response */ public Response trace(URI uri); /** * Makes an HTTP TRACE request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response trace(RequestOptions requestOptions); /** * Makes an HTTP OPTIONS request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response options(String url) throws URISyntaxException; /** * Makes an HTTP OPTIONS request * @param uri the uri against which to make the request * @return the HTTP response */ public Response options(URI uri); /** * Makes an HTTP OPTIONS request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response options(RequestOptions requestOptions); /** * Makes an HTTP PATCH request * @param url the url against which to make the request * @return the HTTP response * @throws URISyntaxException */ public Response patch(String url) throws URISyntaxException; /** * Makes an HTTP PATCH request * @param uri the uri against which to make the request * @return the HTTP response */ public Response patch(URI uri); /** * Makes an HTTP PATCH request * @param requestOptions the options to configure the request * @return the HTTP response */ public Response patch(RequestOptions requestOptions); } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/000077500000000000000000000000001312026620400243605ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/CoercedClientOptions.java000066400000000000000000000033011312026620400312770ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import javax.net.ssl.SSLContext; public class CoercedClientOptions { private final SSLContext sslContext; private final String[] sslProtocols; private final String[] sslCipherSuites; private final boolean forceRedirects; private final boolean followRedirects; private final int connectTimeoutMilliseconds; private final int socketTimeoutMilliseconds; public CoercedClientOptions(SSLContext sslContext, String[] sslProtocols, String[] sslCipherSuites, boolean forceRedirects, boolean followRedirects, int connectTimeoutMilliseconds, int socketTimeoutMilliseconds) { this.sslContext = sslContext; this.sslProtocols = sslProtocols; this.sslCipherSuites = sslCipherSuites; this.forceRedirects = forceRedirects; this.followRedirects = followRedirects; this.connectTimeoutMilliseconds = connectTimeoutMilliseconds; this.socketTimeoutMilliseconds = socketTimeoutMilliseconds; } public SSLContext getSslContext() { return sslContext; } public String[] getSslProtocols() { return sslProtocols; } public String[] getSslCipherSuites() { return sslCipherSuites; } public boolean getForceRedirects() { return forceRedirects; } public boolean getFollowRedirects() { return followRedirects; } public int getConnectTimeoutMilliseconds() { return connectTimeoutMilliseconds; } public int getSocketTimeoutMilliseconds() { return socketTimeoutMilliseconds; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/CoercedRequestOptions.java000066400000000000000000000025551312026620400315230ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.puppetlabs.http.client.HttpMethod; import org.apache.http.Header; import org.apache.http.HttpEntity; import java.net.URI; import java.util.zip.GZIPOutputStream; class CoercedRequestOptions { private final URI uri; private final HttpMethod method; private final Header[] headers; private final HttpEntity body; private final GZIPOutputStream gzipOutputStream; private final byte[] bytesToGzip; public CoercedRequestOptions(URI uri, HttpMethod method, Header[] headers, HttpEntity body, GZIPOutputStream gzipOutputStream, byte[] bytesToGzip) { this.uri = uri; this.method = method; this.headers = headers; this.body = body; this.gzipOutputStream = gzipOutputStream; this.bytesToGzip = bytesToGzip; } public URI getUri() { return uri; } public HttpMethod getMethod() { return method; } public Header[] getHeaders() { return headers; } public HttpEntity getBody() { return body; } public GZIPOutputStream getGzipOutputStream() { return gzipOutputStream; }; public byte[] getBytesToGzip() { return bytesToGzip; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/Deliverable.java000066400000000000000000000001451312026620400274410ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; public interface Deliverable { void deliver(T t); } ExceptionInsertingPipedInputStream.java000066400000000000000000000023701312026620400341450ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/implpackage com.puppetlabs.http.client.impl; import java.io.IOException; import java.io.PipedInputStream; public class ExceptionInsertingPipedInputStream extends PipedInputStream { private final Promise ioExceptionPromise; public ExceptionInsertingPipedInputStream(Promise ioExceptionPromise) { this.ioExceptionPromise = ioExceptionPromise; } private void checkFinalResult() throws IOException { try { IOException ioException = ioExceptionPromise.deref(); if (ioException != null) { throw ioException; } } catch (InterruptedException e) { throw new RuntimeException(e); } } @Override public synchronized int read() throws IOException { int read = super.read(); if (read == -1) { checkFinalResult(); } return read; } @Override public synchronized int read(byte[] b, int off, int len) throws IOException { int read = super.read(b, off, len); if (read == -1) { checkFinalResult(); } return read; } @Override public void close() throws IOException { super.close(); checkFinalResult(); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/FnDeliverable.java000066400000000000000000000004451312026620400277300ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import clojure.lang.IFn; public class FnDeliverable implements Deliverable { private final IFn fn; public FnDeliverable(IFn fn) { this.fn = fn; } @Override public void deliver(T t) { fn.invoke(t); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/IResponseCallback.java000066400000000000000000000002561312026620400305520ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.puppetlabs.http.client.Response; public interface IResponseCallback { Response handleResponse(Response response); } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/JavaClient.java000066400000000000000000000726721312026620400272610ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.codahale.metrics.MetricRegistry; import com.puppetlabs.http.client.ClientOptions; import com.puppetlabs.http.client.CompressType; import com.puppetlabs.http.client.HttpClientException; import com.puppetlabs.http.client.HttpMethod; import com.puppetlabs.http.client.RequestOptions; import com.puppetlabs.http.client.ResponseBodyType; import com.puppetlabs.http.client.impl.metrics.TimerUtils; import org.apache.commons.io.IOUtils; import org.apache.http.Consts; import org.apache.http.Header; import org.apache.http.HttpEntity; import org.apache.http.HttpRequest; import org.apache.http.HttpResponse; import org.apache.http.ParseException; import org.apache.http.ProtocolException; import org.apache.http.client.RedirectStrategy; import org.apache.http.client.config.RequestConfig; import org.apache.http.client.config.RequestConfig.Builder; import org.apache.http.client.methods.HttpDelete; import org.apache.http.client.methods.HttpEntityEnclosingRequestBase; import org.apache.http.client.methods.HttpGet; import org.apache.http.client.methods.HttpHead; import org.apache.http.client.methods.HttpOptions; import org.apache.http.client.methods.HttpPatch; import org.apache.http.client.methods.HttpPost; import org.apache.http.client.methods.HttpPut; import org.apache.http.client.methods.HttpRequestBase; import org.apache.http.client.methods.HttpTrace; import org.apache.http.client.methods.HttpUriRequest; import org.apache.http.client.protocol.HttpClientContext; import org.apache.http.client.protocol.ResponseContentEncoding; import org.apache.http.concurrent.FutureCallback; import org.apache.http.entity.ContentType; import org.apache.http.entity.InputStreamEntity; import org.apache.http.impl.client.DefaultRedirectStrategy; import org.apache.http.impl.client.LaxRedirectStrategy; import org.apache.http.impl.nio.client.CloseableHttpAsyncClient; import org.apache.http.impl.nio.client.HttpAsyncClientBuilder; import org.apache.http.impl.nio.client.HttpAsyncClients; import org.apache.http.message.BasicHeader; import org.apache.http.nio.client.methods.HttpAsyncMethods; import org.apache.http.nio.conn.ssl.SSLIOSessionStrategy; import org.apache.http.nio.entity.NStringEntity; import org.apache.http.protocol.HttpContext; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import javax.net.ssl.SSLContext; import javax.net.ssl.TrustManager; import javax.net.ssl.X509TrustManager; import java.io.IOException; import java.io.InputStream; import java.io.PipedInputStream; import java.io.PipedOutputStream; import java.io.UnsupportedEncodingException; import java.net.URI; import java.nio.charset.UnsupportedCharsetException; import java.security.KeyManagementException; import java.security.NoSuchAlgorithmException; import java.security.cert.CertificateException; import java.security.cert.X509Certificate; import java.util.HashMap; import java.util.Map; import java.util.zip.GZIPOutputStream; public class JavaClient { private static final String PROTOCOL = "TLS"; private static final Logger LOGGER = LoggerFactory.getLogger(JavaClient.class); // Buffer size to use in streams for request gzip compression. This is // somewhat arbitrary but went with the same value as the Apache HTTP // async client uses for chunking input streams for requests: // https://github.com/apache/httpcore/blob/4.4.5/httpcore-nio/src/main/java/org/apache/http/nio/entity/EntityAsyncContentProducer.java#L58 private static int GZIP_BUFFER_SIZE = 4096; private static Header[] prepareHeaders(RequestOptions options, ContentType contentType) { Map result = new HashMap(); Map origHeaders = options.getHeaders(); if (origHeaders == null) { origHeaders = new HashMap(); } for (Map.Entry entry : origHeaders.entrySet()) { result.put(entry.getKey().toLowerCase(), new BasicHeader(entry.getKey(), entry.getValue())); } if (options.getDecompressBody() && (! result.containsKey("accept-encoding"))) { result.put("accept-encoding", new BasicHeader("Accept-Encoding", "gzip, deflate")); } if (options.getCompressRequestBody() == CompressType.GZIP && (! result.containsKey("content-encoding"))) { result.put("content-encoding", new BasicHeader("Content-Encoding", "gzip")); } if (contentType != null) { result.put("content-type", new BasicHeader("Content-Type", contentType.toString())); } return result.values().toArray(new Header[result.size()]); } public static ContentType getContentType (Object body, RequestOptions options) { ContentType contentType = null; Map headers = options.getHeaders(); if (headers != null) { for (Map.Entry entry : headers.entrySet()) { if (entry.getKey().toLowerCase().equals("content-type")) { String contentTypeValue = entry.getValue(); if (contentTypeValue != null && !contentTypeValue.isEmpty()) { try { contentType = ContentType.parse(contentTypeValue); } catch (ParseException e) { throw new HttpClientException("Unable to parse request content type", e); } catch (UnsupportedCharsetException e) { throw new HttpClientException("Unsupported content type charset", e); } // In the case when the caller provides the body as a string, and does not // specify a charset, we choose one for them. There will always be _some_ // charset used to encode the string, and in this case we choose UTF-8 // (instead of letting the underlying Apache HTTP client library // choose ISO-8859-1) because UTF-8 is a more reasonable default. if (contentType.getCharset() == null && body instanceof String) { contentType = ContentType.create(contentType.getMimeType(), Consts.UTF_8); } } } } } return contentType; } private static void throwUnsupportedBodyException(Object body) { throw new HttpClientException("Unsupported body type for request: " + body.getClass() + ". Only InputStream and String are supported."); } private static CoercedRequestOptions coerceRequestOptions(RequestOptions options, HttpMethod method) { URI uri = options.getUri(); if (method == null) { method = HttpMethod.GET; } ContentType contentType = getContentType(options.getBody(), options); Header[] headers = prepareHeaders(options, contentType); HttpEntity body = null; GZIPOutputStream gzipOutputStream = null; Object bodyFromOptions = options.getBody(); byte[] bytesToGzip = null; if ((bodyFromOptions instanceof String) || (bodyFromOptions instanceof InputStream)) { // See comments in the requestWithClient() method about why the // request body is routed through a GZIPOutputStream, // PipedOutputStream, and PipedInputStream in order to achieve // gzip compression. if (options.getCompressRequestBody() == CompressType.GZIP) { PipedInputStream pis = new PipedInputStream(GZIP_BUFFER_SIZE); PipedOutputStream pos = new PipedOutputStream(); try { pos.connect(pis); gzipOutputStream = new GZIPOutputStream(pos, GZIP_BUFFER_SIZE); body = new InputStreamEntity(pis); } catch (IOException ioe) { throw new HttpClientException( "Error setting up gzip stream for request", ioe); } if (bodyFromOptions instanceof String) { String bodyAsString = (String) bodyFromOptions; if (contentType != null) { bytesToGzip = bodyAsString.getBytes(contentType.getCharset()); } else { bytesToGzip = bodyAsString.getBytes(); } } } else if (bodyFromOptions instanceof String) { String originalBody = (String) bodyFromOptions; if (contentType != null) { body = new NStringEntity(originalBody, contentType); } else { try { body = new NStringEntity(originalBody); } catch (UnsupportedEncodingException e) { throw new HttpClientException( "Unable to create request body", e); } } } else { body = new InputStreamEntity((InputStream) bodyFromOptions); } } else if (bodyFromOptions != null) { throwUnsupportedBodyException(bodyFromOptions); } return new CoercedRequestOptions(uri, method, headers, body, gzipOutputStream, bytesToGzip); } public static CoercedClientOptions coerceClientOptions(ClientOptions options) { SSLContext sslContext = null; if (options.getSslContext() != null) { sslContext = options.getSslContext(); } else if (options.getInsecure()) { sslContext = getInsecureSslContext(); } String[] sslProtocols = null; if (options.getSslProtocols() != null) { sslProtocols = options.getSslProtocols(); } else { sslProtocols = ClientOptions.DEFAULT_SSL_PROTOCOLS; } String[] sslCipherSuites = null; if (options.getSslCipherSuites() != null) { sslCipherSuites = options.getSslCipherSuites(); } boolean forceRedirects = options.getForceRedirects(); boolean followRedirects = options.getFollowRedirects(); int connectTimeoutMilliseconds = options.getConnectTimeoutMilliseconds(); int socketTimeoutMilliseconds = options.getSocketTimeoutMilliseconds(); return new CoercedClientOptions(sslContext, sslProtocols, sslCipherSuites, forceRedirects, followRedirects, connectTimeoutMilliseconds, socketTimeoutMilliseconds); } private static SSLContext getInsecureSslContext() { SSLContext context = null; try { context = SSLContext.getInstance(PROTOCOL); } catch (NoSuchAlgorithmException e) { throw new HttpClientException("Unable to construct HTTP context", e); } try { context.init(null, new TrustManager[] { new X509TrustManager() { public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException { // Always trust } public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException { // Always trust } }}, null); } catch (KeyManagementException e) { throw new HttpClientException("Unable to initialize insecure SSL context", e); } return context; } private static void completeResponse(ResponseDeliveryDelegate responseDeliveryDelegate, RequestOptions requestOptions, IResponseCallback callback, HttpResponse httpResponse, HttpContext httpContext) { try { Map headers = new HashMap<>(); for (Header h : httpResponse.getAllHeaders()) { headers.put(h.getName().toLowerCase(), h.getValue()); } String origContentEncoding = headers.get("content-encoding"); if (requestOptions.getDecompressBody()) { new ResponseContentEncoding().process(httpResponse, httpContext); } Object body = null; HttpEntity entity = httpResponse.getEntity(); if (entity != null) { body = entity.getContent(); } ContentType contentType = null; String contentTypeValue = headers.get("content-type"); if (contentTypeValue != null && !contentTypeValue.isEmpty()) { contentType = ContentType.parse(contentTypeValue); } if (requestOptions.getAs() == ResponseBodyType.TEXT) { body = coerceBodyType((InputStream) body, requestOptions.getAs(), contentType); } responseDeliveryDelegate.deliverResponse(requestOptions, origContentEncoding, body, headers, httpResponse.getStatusLine().getStatusCode(), contentType, callback); } catch (Exception e) { responseDeliveryDelegate.deliverResponse(requestOptions, e, callback); } } private static void executeWithConsumer(final CloseableHttpAsyncClient client, final FutureCallback futureCallback, final HttpRequestBase request, final MetricRegistry metricRegistry, final String[] metricId, final String metricNamespace, final boolean enableURLMetrics) { /* * Create an Apache AsyncResponseConsumer that will return the response to us as soon as it is available, * then send the response body asynchronously */ final StreamingAsyncResponseConsumer consumer = new StreamingAsyncResponseConsumer(new Deliverable() { @Override public void deliver(HttpResponse httpResponse) { futureCallback.completed(httpResponse); } }); /* * Normally the consumer returns the response as soon as it is available using the deliver() callback (above) * which delegates to the supplied futureCallback. * * If an error occurs early in the request, the consumer may not get a chance to deliver the response. This * streamingCompleteCallback wraps the supplied futureCallback and ensures: * - The supplied futureCallback is always eventually called even in error states * - Any exception that occurs during stream processing (after the response has been returned) is propagated * back to the client using the setFinalResult() method. */ final FutureCallback streamingCompleteCallback = new FutureCallback() { @Override public void completed(HttpResponse httpResponse) { consumer.setFinalResult(null); futureCallback.completed(httpResponse); } @Override public void failed(Exception e) { if (e instanceof IOException) { consumer.setFinalResult((IOException) e); } else { consumer.setFinalResult(new IOException(e)); } futureCallback.failed(e); } @Override public void cancelled() { consumer.setFinalResult(null); futureCallback.cancelled(); } }; TimedFutureCallback timedStreamingCompleteCallback = new TimedFutureCallback<>(streamingCompleteCallback, TimerUtils.startFullResponseTimers(metricRegistry, request, metricId, metricNamespace, enableURLMetrics)); client.execute(HttpAsyncMethods.create(request), consumer, timedStreamingCompleteCallback); } private static void gzipRequestPayload( GZIPOutputStream gzipOutputStream, byte[] bytesToGzip, Object requestBody) { try { if (bytesToGzip != null) { gzipOutputStream.write(bytesToGzip); } else { if (requestBody instanceof InputStream) { InputStream requestInputStream = (InputStream) requestBody; byte[] byteBuffer = new byte[GZIP_BUFFER_SIZE]; try { IOUtils.copyLarge(requestInputStream, gzipOutputStream, byteBuffer); } finally { requestInputStream.close(); } } else { throwUnsupportedBodyException(requestBody); } } // IOExceptions may be thrown either during the IOUtils.copyLarge() // call above or during the close() call to the GZIPOutputStream below. // The GZIPOutputStream object is backed by a PipedOutputStream object // which is connected to a PipedInputStream object. Most likely, any // IOExceptions thrown up to this level would be due to the underlying // PipedInputStream being closed prematurely. In those cases, the // Apache HTTP Async library should detect the failure while processing // the request and deliver an appropriate "failure" callback as the // result for the request. The IOExceptions are caught and not rethrown // here so that the client can still receive the "failure" callback from // the Apache HTTP Async library later on. The exceptions are still // logged here at a debug level for troubleshooting purposes. } catch (IOException ioe) { LOGGER.debug("Error writing gzip request body", ioe); } finally { try { gzipOutputStream.close(); } catch (IOException ioe) { LOGGER.debug("Error closing gzip request stream", ioe); } } } public static void requestWithClient(final RequestOptions requestOptions, final HttpMethod method, final IResponseCallback callback, final CloseableHttpAsyncClient client, final ResponseDeliveryDelegate responseDeliveryDelegate, final MetricRegistry registry, final String metricNamespace, final boolean enableURLMetrics) { final CoercedRequestOptions coercedRequestOptions = coerceRequestOptions(requestOptions, method); final HttpRequestBase request = constructRequest(coercedRequestOptions.getMethod(), coercedRequestOptions.getUri(), coercedRequestOptions.getBody()); request.setHeaders(coercedRequestOptions.getHeaders()); final HttpContext httpContext = HttpClientContext.create(); final FutureCallback futureCallback = new FutureCallback() { @Override public void completed(HttpResponse httpResponse) { completeResponse(responseDeliveryDelegate, requestOptions, callback, httpResponse, httpContext); } @Override public void failed(Exception e) { responseDeliveryDelegate.deliverResponse(requestOptions, e, callback); } @Override public void cancelled() { responseDeliveryDelegate.deliverResponse(requestOptions, new HttpClientException("Request cancelled"), callback); } }; final String[] metricId = requestOptions.getMetricId(); if (requestOptions.getAs() == ResponseBodyType.UNBUFFERED_STREAM) { executeWithConsumer(client, futureCallback, request, registry, metricId, metricNamespace, enableURLMetrics); } else { TimedFutureCallback timedFutureCallback = new TimedFutureCallback<>(futureCallback, TimerUtils.startFullResponseTimers(registry, request, metricId, metricNamespace, enableURLMetrics)); client.execute(request, timedFutureCallback); } // The approach used for gzip-compressing the request payload is far from // ideal here. The approach involves reading the bytes from the supplied // request body, redirecting those through a JDK GZIPOutputStream object // to compress them, and then piping those back into in InputStream that // the Apache Async HTTP layer reads from in order to get the bytes // to transmit in the HTTP request. The JDK apparently has no built-in // functionality for gzip-compressing a source byte array or InputStream // back into a separate InputStream that the Apache Async HTTP layer // could use. // // A better approach would probably be to do something like one of the // approaches discussed in http://stackoverflow.com/questions/11036280/compress-an-inputstream-with-gzip. // For example, the InputStream given to the Apache Async HTTP layer // could be wrapped with a FilterInputStream which gzip-compresses // bytes on the fly as the Apache Async HTTP layer asks for them. The // approaches on that thread are pretty involved, though, and appear to // have liberally copied content from the JDK source. A clean-room // implementation would probably be better but would also likely // require a fair bit of testing to ensure that it produces good gzip // content under varying read scenarios. // // The approach being used for now requires writing through the // GZIPOutputStream and underlying PipedOutputStream from the thread on // which the HTTP request is made. The connected PipedInputStream is // then read from a separate thread, one of the Apache HTTP Async IO // worker threads -- hopefully avoiding the possibility of a deadlock // in the process. // // For requests that provide an InputStream as a source argument, it // would also probably be more performant to do the GZIPOutputStream // writing from a separate thread and would give an AsyncHttpClient // requestor the ability to do other work while the source InputStream // is being read and compressed. As a simplification for now, this // implementation doesn't spin up a separate thread (or thread pool) // for performing gzip compression. GZIPOutputStream gzipOutputStream = coercedRequestOptions.getGzipOutputStream(); if (gzipOutputStream != null) { gzipRequestPayload(gzipOutputStream, coercedRequestOptions.getBytesToGzip(), requestOptions.getBody()); } } public static CloseableHttpAsyncClient createClient(ClientOptions clientOptions) { CoercedClientOptions coercedOptions = coerceClientOptions(SslUtils.configureSsl(clientOptions)); HttpAsyncClientBuilder clientBuilder = HttpAsyncClients.custom(); if (coercedOptions.getSslContext() != null) { clientBuilder.setSSLStrategy( new SSLIOSessionStrategy(coercedOptions.getSslContext(), coercedOptions.getSslProtocols(), coercedOptions.getSslCipherSuites(), SSLIOSessionStrategy.BROWSER_COMPATIBLE_HOSTNAME_VERIFIER)); } RedirectStrategy redirectStrategy; if (!coercedOptions.getFollowRedirects()) { redirectStrategy = new RedirectStrategy() { @Override public boolean isRedirected(HttpRequest httpRequest, HttpResponse httpResponse, HttpContext httpContext) throws ProtocolException { return false; } @Override public HttpUriRequest getRedirect(HttpRequest httpRequest, HttpResponse httpResponse, HttpContext httpContext) throws ProtocolException { return null; } }; } else if (coercedOptions.getForceRedirects()) { redirectStrategy = new LaxRedirectStrategy(); } else { redirectStrategy = new DefaultRedirectStrategy(); } clientBuilder.setRedirectStrategy(redirectStrategy); RequestConfig requestConfig = getRequestConfig(coercedOptions); if (requestConfig != null) { clientBuilder.setDefaultRequestConfig(requestConfig); } CloseableHttpAsyncClient client = clientBuilder.build(); client.start(); return client; } private static RequestConfig getRequestConfig (CoercedClientOptions options) { RequestConfig config = null; int connectTimeoutMilliseconds = options.getConnectTimeoutMilliseconds(); int socketTimeoutMilliseconds = options.getSocketTimeoutMilliseconds(); if (connectTimeoutMilliseconds >= 0 || socketTimeoutMilliseconds >= 0) { Builder requestConfigBuilder = RequestConfig.custom(); if (connectTimeoutMilliseconds >= 0) { requestConfigBuilder.setConnectTimeout (connectTimeoutMilliseconds); } if (socketTimeoutMilliseconds >= 0) { requestConfigBuilder.setSocketTimeout (socketTimeoutMilliseconds); } config = requestConfigBuilder.build(); } return config; } private static HttpRequestBase constructRequest(HttpMethod httpMethod, URI uri, HttpEntity body) { switch (httpMethod) { case GET: return requestWithNoBody(new HttpGet(uri), body, httpMethod); case HEAD: return requestWithNoBody(new HttpHead(uri), body, httpMethod); case POST: return requestWithBody(new HttpPost(uri), body); case PUT: return requestWithBody(new HttpPut(uri), body); case DELETE: return requestWithNoBody(new HttpDelete(uri), body, httpMethod); case TRACE: return requestWithNoBody(new HttpTrace(uri), body, httpMethod); case OPTIONS: return requestWithNoBody(new HttpOptions(uri), body, httpMethod); case PATCH: return requestWithBody(new HttpPatch(uri), body); default: throw new HttpClientException("Unable to construct request for:" + httpMethod + ", " + uri.toString(), null); } } private static HttpRequestBase requestWithBody(HttpEntityEnclosingRequestBase request, HttpEntity body) { if (body != null) { request.setEntity(body); } return request; } private static HttpRequestBase requestWithNoBody(HttpRequestBase request, Object body, HttpMethod httpMethod) { if (body != null) { throw new HttpClientException("Request of type " + httpMethod + " does not support 'body'!"); } return request; } public static Object coerceBodyType(InputStream body, ResponseBodyType as, ContentType contentType) { Object response = null; switch (as) { case TEXT: String charset = "UTF-8"; if ((contentType != null) && (contentType.getCharset() != null)) { charset = contentType.getCharset().name(); } try { if (body == null){ response = ""; } else{ response = IOUtils.toString(body, charset); } } catch (IOException e) { throw new HttpClientException("Unable to read body as string", e); } try { if (body != null){ body.close(); } } catch (IOException e) { throw new HttpClientException( "Unable to close response stream", e); } break; default: throw new HttpClientException("Unsupported body type: " + as); } return response; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/JavaResponseDeliveryDelegate.java000066400000000000000000000034321312026620400327640ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.puppetlabs.http.client.RequestOptions; import com.puppetlabs.http.client.Response; import org.apache.http.entity.ContentType; import java.util.Map; public final class JavaResponseDeliveryDelegate implements ResponseDeliveryDelegate { private final Promise promise; public JavaResponseDeliveryDelegate(Promise promise) { this.promise = promise; } private void deliverResponse(Response response, RequestOptions requestOptions, IResponseCallback callback) { if (callback != null) { try { promise.deliver(callback.handleResponse(response)); } catch (Exception e) { promise.deliver(new Response(requestOptions, e)); } } else { promise.deliver(response); } } @Override public void deliverResponse(RequestOptions requestOptions, String origContentEncoding, Object body, Map headers, int statusCode, ContentType contentType, IResponseCallback callback) { Response response = new Response(requestOptions, origContentEncoding, body, headers, statusCode, contentType); deliverResponse(response, requestOptions, callback); } @Override public void deliverResponse(RequestOptions requestOptions, Exception e, IResponseCallback callback) { deliverResponse(new Response(requestOptions, e), requestOptions, callback); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/PersistentAsyncHttpClient.java000066400000000000000000000107301312026620400323610ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.codahale.metrics.MetricRegistry; import com.puppetlabs.http.client.Response; import com.puppetlabs.http.client.RequestOptions; import com.puppetlabs.http.client.HttpMethod; import com.puppetlabs.http.client.AsyncHttpClient; import org.apache.http.impl.nio.client.CloseableHttpAsyncClient; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; public class PersistentAsyncHttpClient implements AsyncHttpClient { private CloseableHttpAsyncClient client; private MetricRegistry metricRegistry; private String metricNamespace; private boolean enableURLMetrics; public PersistentAsyncHttpClient(CloseableHttpAsyncClient client, MetricRegistry metricRegistry, String metricNamespace, boolean enableURLMetrics) { this.client = client; this.metricRegistry = metricRegistry; this.metricNamespace = metricNamespace; this.enableURLMetrics = enableURLMetrics; } public void close() throws IOException { client.close(); } public MetricRegistry getMetricRegistry() { return metricRegistry; } public String getMetricNamespace() { return metricNamespace; } private Promise request(RequestOptions requestOptions, HttpMethod method) { final Promise promise = new Promise<>(); final JavaResponseDeliveryDelegate responseDelivery = new JavaResponseDeliveryDelegate(promise); JavaClient.requestWithClient(requestOptions, method, null, client, responseDelivery, metricRegistry, metricNamespace, enableURLMetrics); return promise; } public Promise get(String url) throws URISyntaxException { return get(new URI(url)); } public Promise get(URI uri) { return get(new RequestOptions(uri)); } public Promise get(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.GET); } public Promise head(String url) throws URISyntaxException { return head(new URI(url)); } public Promise head(URI uri) { return head(new RequestOptions(uri)); } public Promise head(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.HEAD); } public Promise post(String url) throws URISyntaxException { return post(new URI(url)); } public Promise post(URI uri) { return post(new RequestOptions(uri)); } public Promise post(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.POST); } public Promise put(String url) throws URISyntaxException { return put(new URI(url)); } public Promise put(URI uri) { return put(new RequestOptions(uri)); } public Promise put(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.PUT); } public Promise delete(String url) throws URISyntaxException { return delete(new URI(url)); } public Promise delete(URI uri) { return delete(new RequestOptions(uri)); } public Promise delete(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.DELETE); } public Promise trace(String url) throws URISyntaxException { return trace(new URI(url)); } public Promise trace(URI uri) { return trace(new RequestOptions(uri)); } public Promise trace(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.TRACE); } public Promise options(String url) throws URISyntaxException { return options(new URI(url)); } public Promise options(URI uri) { return options(new RequestOptions(uri)); } public Promise options(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.OPTIONS); } public Promise patch(String url) throws URISyntaxException { return patch(new URI(url)); } public Promise patch(URI uri) { return patch(new RequestOptions(uri)); } public Promise patch(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.PATCH); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/PersistentSyncHttpClient.java000066400000000000000000000116741312026620400322300ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.codahale.metrics.MetricRegistry; import com.puppetlabs.http.client.HttpClientException; import com.puppetlabs.http.client.Response; import com.puppetlabs.http.client.RequestOptions; import com.puppetlabs.http.client.HttpMethod; import com.puppetlabs.http.client.SyncHttpClient; import org.apache.http.impl.nio.client.CloseableHttpAsyncClient; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; public class PersistentSyncHttpClient implements SyncHttpClient { private CloseableHttpAsyncClient client; private MetricRegistry metricRegistry; private String metricNamespace; private boolean enableURLMetrics; private static final Logger LOGGER = LoggerFactory.getLogger(PersistentSyncHttpClient.class); public PersistentSyncHttpClient(CloseableHttpAsyncClient client, MetricRegistry metricRegistry, String metricNamespace, boolean enableURLMetrics) { this.client = client; this.metricRegistry = metricRegistry; this.metricNamespace = metricNamespace; this.enableURLMetrics = enableURLMetrics; } private static void logAndRethrow(String msg, Throwable t) { LOGGER.error(msg, t); throw new HttpClientException(msg, t); } public MetricRegistry getMetricRegistry() { return metricRegistry; } public String getMetricNamespace() { return metricNamespace; } public Response request(RequestOptions requestOptions, HttpMethod method) { final Promise promise = new Promise<>(); final JavaResponseDeliveryDelegate responseDelivery = new JavaResponseDeliveryDelegate(promise); JavaClient.requestWithClient(requestOptions, method, null, client, responseDelivery, metricRegistry, metricNamespace, enableURLMetrics); Response response = null; try { response = promise.deref(); if (response.getError() != null) { logAndRethrow("Error executing http request", response.getError()); } } catch (InterruptedException e) { logAndRethrow("Error while waiting for http response", e); } return response; } public void close() throws IOException { client.close(); } public Response get(String url) throws URISyntaxException { return get(new URI(url)); } public Response get(URI uri) { return get(new RequestOptions(uri)); } public Response get(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.GET); } public Response head(String url) throws URISyntaxException { return head(new URI(url)); } public Response head(URI uri) { return head(new RequestOptions(uri)); } public Response head(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.HEAD); } public Response post(String url) throws URISyntaxException { return post(new URI(url)); } public Response post(URI uri) { return post(new RequestOptions(uri)); } public Response post(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.POST); } public Response put(String url) throws URISyntaxException { return put(new URI(url)); } public Response put(URI uri) { return put(new RequestOptions(uri)); } public Response put(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.PUT); } public Response delete(String url) throws URISyntaxException { return delete(new URI(url)); } public Response delete(URI uri) { return delete(new RequestOptions(uri)); } public Response delete(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.DELETE); } public Response trace(String url) throws URISyntaxException { return trace(new URI(url)); } public Response trace(URI uri) { return trace(new RequestOptions(uri)); } public Response trace(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.TRACE); } public Response options(String url) throws URISyntaxException { return options(new URI(url)); } public Response options(URI uri) { return options(new RequestOptions(uri)); } public Response options(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.OPTIONS); } public Response patch(String url) throws URISyntaxException { return patch(new URI(url)); } public Response patch(URI uri) { return patch(new RequestOptions(uri)); } public Response patch(RequestOptions requestOptions) { return request(requestOptions, HttpMethod.PATCH); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/Promise.java000066400000000000000000000011771312026620400266470ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import java.util.concurrent.CountDownLatch; public class Promise implements Deliverable { private final CountDownLatch latch; private T value = null; public Promise() { latch = new CountDownLatch(1); } public synchronized void deliver(T t) { if (value != null) { throw new IllegalStateException("Attempting to deliver value to a promise that has already been realized!"); } value = t; latch.countDown(); } public T deref() throws InterruptedException { latch.await(); return value; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/ResponseDeliveryDelegate.java000066400000000000000000000012621312026620400321610ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.puppetlabs.http.client.RequestOptions; import org.apache.http.entity.ContentType; public interface ResponseDeliveryDelegate { void deliverResponse(RequestOptions requestOptions, String origContentEncoding, Object body, java.util.Map headers, int statusCode, ContentType contentType, IResponseCallback callback); void deliverResponse(RequestOptions requestOptions, Exception e, IResponseCallback callback); } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/SslUtils.java000066400000000000000000000062431312026620400270120ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.puppetlabs.ssl_utils.SSLUtils; import com.puppetlabs.http.client.HttpClientException; import com.puppetlabs.http.client.ClientOptions; import com.puppetlabs.http.client.Sync; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.io.FileReader; import java.io.IOException; import java.security.KeyManagementException; import java.security.KeyStoreException; import java.security.NoSuchAlgorithmException; import java.security.UnrecoverableKeyException; import java.security.cert.CertificateException; public class SslUtils { private static final Logger LOGGER = LoggerFactory.getLogger(SslUtils.class); private static void logAndRethrow(String msg, Throwable t) { LOGGER.error(msg, t); throw new HttpClientException(msg, t); } public static ClientOptions configureSsl(ClientOptions options) { if (options.getSslContext() != null) { return options; } if ((options.getSslCert() != null) && (options.getSslKey() != null) && (options.getSslCaCert() != null)) { try { options.setSslContext( SSLUtils.pemsToSSLContext( new FileReader(options.getSslCert()), new FileReader(options.getSslKey()), new FileReader(options.getSslCaCert())) ); } catch (KeyStoreException e) { logAndRethrow("Error while configuring SSL", e); } catch (CertificateException e) { logAndRethrow("Error while configuring SSL", e); } catch (IOException e) { logAndRethrow("Error while configuring SSL", e); } catch (NoSuchAlgorithmException e) { logAndRethrow("Error while configuring SSL", e); } catch (KeyManagementException e) { logAndRethrow("Error while configuring SSL", e); } catch (UnrecoverableKeyException e) { logAndRethrow("Error while configuring SSL", e); } options.setSslCert(null); options.setSslKey(null); options.setSslCaCert(null); return options; } if (options.getSslCaCert() != null) { try { options.setSslContext( SSLUtils.caCertPemToSSLContext( new FileReader(options.getSslCaCert())) ); } catch (KeyStoreException e) { logAndRethrow("Error while configuring SSL", e); } catch (CertificateException e) { logAndRethrow("Error while configuring SSL", e); } catch (IOException e) { logAndRethrow("Error while configuring SSL", e); } catch (NoSuchAlgorithmException e) { logAndRethrow("Error while configuring SSL", e); } catch (KeyManagementException e) { logAndRethrow("Error while configuring SSL", e); } options.setSslCaCert(null); return options; } return options; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/StreamingAsyncResponseConsumer.java000066400000000000000000000040631312026620400334100ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import org.apache.http.HttpResponse; import org.apache.http.entity.BasicHttpEntity; import org.apache.http.nio.IOControl; import org.apache.http.nio.client.methods.AsyncByteConsumer; import org.apache.http.protocol.HttpContext; import java.io.IOException; import java.io.PipedInputStream; import java.io.PipedOutputStream; import java.nio.ByteBuffer; public class StreamingAsyncResponseConsumer extends AsyncByteConsumer { private volatile HttpResponse response; private volatile PipedOutputStream pos; private volatile Deliverable promise; private volatile Promise ioExceptionPromise = new Promise<>(); public void setFinalResult(IOException ioException) { ioExceptionPromise.deliver(ioException); } public StreamingAsyncResponseConsumer(Deliverable promise) { this.promise = promise; } @Override protected void onResponseReceived(final HttpResponse response) throws IOException { PipedInputStream pis = new ExceptionInsertingPipedInputStream(ioExceptionPromise); pos = new PipedOutputStream(); pos.connect(pis); ((BasicHttpEntity) response.getEntity()).setContent(pis); this.response = response; promise.deliver(response); } @Override protected void onByteReceived(final ByteBuffer buf, final IOControl ioctrl) throws IOException { while (buf.hasRemaining()) { byte[] bs = new byte[buf.remaining()]; buf.get(bs); pos.write(bs); } } @Override protected void releaseResources() { super.releaseResources(); this.response = null; this.promise = null; try { if (pos != null) { this.pos.close(); this.pos = null; } } catch (IOException e) { throw new IllegalStateException(e); } } @Override protected HttpResponse buildResult(final HttpContext context) { return response; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/TimedFutureCallback.java000066400000000000000000000020611312026620400310740ustar00rootroot00000000000000package com.puppetlabs.http.client.impl; import com.codahale.metrics.Timer; import org.apache.http.concurrent.FutureCallback; import java.util.ArrayList; public final class TimedFutureCallback implements FutureCallback { private final FutureCallback delegate; private final ArrayList timerContexts; public TimedFutureCallback(FutureCallback delegate, ArrayList timerContexts) { this.delegate = delegate; this.timerContexts = timerContexts; } public void completed(T result) { stopTimerContexts(); delegate.completed(result); } public void failed(Exception ex) { stopTimerContexts(); delegate.failed(ex); } public void cancelled() { stopTimerContexts(); delegate.cancelled(); } private void stopTimerContexts() { if (timerContexts != null) { for (Timer.Context timerContext : timerContexts) { timerContext.stop(); } } } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metrics/000077500000000000000000000000001312026620400260265ustar00rootroot00000000000000CategoryClientTimerMetricFilter.java000066400000000000000000000012261312026620400350420ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metricspackage com.puppetlabs.http.client.impl.metrics; import com.codahale.metrics.Metric; import com.codahale.metrics.MetricFilter; import com.puppetlabs.http.client.metrics.ClientTimer; import com.puppetlabs.http.client.metrics.Metrics; public class CategoryClientTimerMetricFilter implements MetricFilter { private final Metrics.MetricCategory category; public CategoryClientTimerMetricFilter(Metrics.MetricCategory category) { this.category = category; } @Override public boolean matches(String s, Metric metric) { return metric instanceof ClientTimer && ((ClientTimer) metric).isCategory(category); } } MetricIdClientTimerFilter.java000066400000000000000000000012441312026620400336210ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metricspackage com.puppetlabs.http.client.impl.metrics; import com.codahale.metrics.Metric; import com.codahale.metrics.MetricFilter; import com.puppetlabs.http.client.metrics.MetricIdClientTimer; import java.util.List; public class MetricIdClientTimerFilter implements MetricFilter { private final List metricId; public MetricIdClientTimerFilter(List metricId) { this.metricId = metricId; } @Override public boolean matches(String s, Metric metric) { return metric.getClass().equals(MetricIdClientTimer.class) && ((MetricIdClientTimer) metric). getMetricId().equals(metricId); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metrics/TimerMetricData.java000066400000000000000000000023401312026620400317060ustar00rootroot00000000000000package com.puppetlabs.http.client.impl.metrics; import com.puppetlabs.http.client.metrics.ClientTimer; import java.util.concurrent.TimeUnit; public class TimerMetricData { public static TimerMetricData fromTimer(ClientTimer timer) { Double mean = timer.getSnapshot().getMean(); Long count = timer.getCount(); Long meanMillis = TimeUnit.NANOSECONDS.toMillis(mean.longValue()); return new TimerMetricData( timer.getMetricName(), meanMillis, count, count * meanMillis); } private final String metricName; private final Long meanMillis; private final Long count; private final Long aggregate; public TimerMetricData(String metricName, Long meanMillis, Long count, Long aggregate) { this.metricName = metricName; this.meanMillis = meanMillis; this.count = count; this.aggregate = aggregate; } public String getMetricName() { return metricName; } public Long getMeanMillis() { return meanMillis; } public Long getCount() { return count; } public Long getAggregate() { return aggregate; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metrics/TimerUtils.java000066400000000000000000000136671312026620400310070ustar00rootroot00000000000000package com.puppetlabs.http.client.impl.metrics; import com.codahale.metrics.Metric; import com.codahale.metrics.MetricRegistry; import com.codahale.metrics.Timer; import com.puppetlabs.http.client.metrics.ClientTimer; import com.puppetlabs.http.client.metrics.MetricIdClientTimer; import com.puppetlabs.http.client.metrics.Metrics; import com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer; import com.puppetlabs.http.client.metrics.UrlClientTimer; import org.apache.http.HttpRequest; import org.apache.http.RequestLine; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.net.URISyntaxException; import java.util.ArrayList; import java.util.Map; public class TimerUtils { private static final Logger LOGGER = LoggerFactory.getLogger(TimerUtils.class); private static ClientTimer getOrAddTimer(MetricRegistry metricRegistry, String name, ClientTimer newTimer) { final Map metrics = metricRegistry.getMetrics(); final Metric metric = metrics.get(name); if ( metric instanceof ClientTimer ) { return (ClientTimer) metric; } else if ( metric == null ) { try { return metricRegistry.register(name, newTimer); } catch (IllegalArgumentException e) { final Metric added = metricRegistry.getMetrics().get(name); if ( added instanceof ClientTimer ) { return (ClientTimer) added; } } } throw new IllegalArgumentException(name +" is already used for a different type of metric"); } private static ArrayList startFullResponseMetricIdTimers(MetricRegistry registry, String[] metricId, String metricPrefix) { ArrayList timerContexts = new ArrayList<>(); for (int i = 0; i < metricId.length; i++) { ArrayList currentId = new ArrayList<>(); for (int j = 0; j <= i; j++) { currentId.add(metricId[j]); } ArrayList currentIdWithNamespace = new ArrayList<>(); currentIdWithNamespace.add(Metrics.NAMESPACE_METRIC_ID); currentIdWithNamespace.addAll(currentId); currentIdWithNamespace.add(Metrics.NAMESPACE_FULL_RESPONSE); String metric_name = MetricRegistry.name(metricPrefix, currentIdWithNamespace.toArray(new String[currentIdWithNamespace.size()])); ClientTimer timer = new MetricIdClientTimer(metric_name, currentId, Metrics.MetricType.FULL_RESPONSE); timerContexts.add(getOrAddTimer(registry, metric_name, timer).time()); } return timerContexts; } private static ArrayList startFullResponseUrlTimers(MetricRegistry registry, HttpRequest request, String metricPrefix, boolean enableURLMetrics) { ArrayList timerContexts = new ArrayList<>(); if (enableURLMetrics) { try { final RequestLine requestLine = request.getRequestLine(); final String strippedUrl = Metrics.urlToMetricUrl(requestLine.getUri()); final String method = requestLine.getMethod(); final String urlName = MetricRegistry.name(metricPrefix, Metrics.NAMESPACE_URL, strippedUrl, Metrics.NAMESPACE_FULL_RESPONSE); final String urlAndMethodName = MetricRegistry.name(metricPrefix, Metrics.NAMESPACE_URL_AND_METHOD, strippedUrl, method, Metrics.NAMESPACE_FULL_RESPONSE); ClientTimer urlTimer = new UrlClientTimer(urlName, strippedUrl, Metrics.MetricType.FULL_RESPONSE); timerContexts.add(getOrAddTimer(registry, urlName, urlTimer).time()); ClientTimer urlMethodTimer = new UrlAndMethodClientTimer(urlAndMethodName, strippedUrl, method, Metrics.MetricType.FULL_RESPONSE); timerContexts.add(getOrAddTimer(registry, urlAndMethodName, urlMethodTimer).time()); } catch (URISyntaxException e) { // this shouldn't be possible LOGGER.warn("Could not build URI out of the request URI. Will not create URI timers. " + "We recommend you read http://www.stilldrinking.com/programming-sucks. " + "'now all your snowflakes are urine and you can't even find the cat.'"); } } return timerContexts; } public static ArrayList startFullResponseTimers(MetricRegistry clientRegistry, HttpRequest request, String[] metricId, String metricNamespace, boolean enableURLMetrics) { if (clientRegistry != null) { ArrayList urlTimerContexts = startFullResponseUrlTimers(clientRegistry, request, metricNamespace, enableURLMetrics); ArrayList allTimerContexts = new ArrayList<>(urlTimerContexts); if (metricId != null) { ArrayList metricIdTimers = startFullResponseMetricIdTimers(clientRegistry, metricId, metricNamespace); allTimerContexts.addAll(metricIdTimers); } return allTimerContexts; } else { return null; } } } UrlAndMethodClientTimerFilter.java000066400000000000000000000013741312026620400344530ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metricspackage com.puppetlabs.http.client.impl.metrics; import com.codahale.metrics.Metric; import com.puppetlabs.http.client.metrics.UrlAndMethodClientTimer; public class UrlAndMethodClientTimerFilter extends UrlClientTimerFilter { private final String method; public UrlAndMethodClientTimerFilter(String url, String method) { super(url); this.method = method; } @Override public boolean matches(String s, Metric metric) { if (metric.getClass().equals(UrlAndMethodClientTimer.class)) { UrlAndMethodClientTimer timer = (UrlAndMethodClientTimer) metric; return timer.getMethod().equals(this.method) && timer.getUrl().equals(this.getUrl()); } return false; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/impl/metrics/UrlClientTimerFilter.java000066400000000000000000000012041312026620400327360ustar00rootroot00000000000000package com.puppetlabs.http.client.impl.metrics; import com.codahale.metrics.Metric; import com.codahale.metrics.MetricFilter; import com.puppetlabs.http.client.metrics.UrlClientTimer; public class UrlClientTimerFilter implements MetricFilter { private final String url; public UrlClientTimerFilter(String url) { this.url = url; } protected String getUrl() { return url; } @Override public boolean matches(String s, Metric metric) { return metric.getClass().equals(UrlClientTimer.class) && ((UrlClientTimer) metric). getUrl().equals(url); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/000077500000000000000000000000001312026620400250655ustar00rootroot00000000000000clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/ClientMetricData.java000066400000000000000000000012201312026620400310770ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.puppetlabs.http.client.impl.metrics.TimerMetricData; public abstract class ClientMetricData { private final TimerMetricData timerMetricData; ClientMetricData(TimerMetricData timerMetricData) { this.timerMetricData = timerMetricData; } public String getMetricName() { return timerMetricData.getMetricName(); } public Long getCount() { return timerMetricData.getCount(); } public Long getMean() { return timerMetricData.getMeanMillis(); } public Long getAggregate() { return timerMetricData.getAggregate(); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/ClientMetricDataContainer.java000066400000000000000000000017131312026620400327510ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import java.util.List; public class ClientMetricDataContainer { private final List urlData; private final List urlAndMethodData; private final List metricIdData; public ClientMetricDataContainer(List urlTimers, List urlAndMethodData, List metricIdData) { this.urlData = urlTimers; this.urlAndMethodData = urlAndMethodData; this.metricIdData = metricIdData; } public List getUrlData() { return urlData; } public List getUrlAndMethodData() { return urlAndMethodData; } public List getMetricIdData() { return metricIdData; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/ClientTimer.java000066400000000000000000000011471312026620400301520ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.codahale.metrics.Timer; public abstract class ClientTimer extends Timer { private final String metricName; private final Metrics.MetricType metricType; ClientTimer(String metricName, Metrics.MetricType metricType) { super(); this.metricName = metricName; this.metricType = metricType; } public String getMetricName() { return metricName; } public Metrics.MetricType getMetricType() { return metricType; } public abstract boolean isCategory(Metrics.MetricCategory category); } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/ClientTimerContainer.java000066400000000000000000000016521312026620400320160ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import java.util.List; public class ClientTimerContainer { private final List urlTimers; private final List urlAndMethodTimers; private final List metricIdTimers; public ClientTimerContainer(List urlTimers, List urlAndMethodTimers, List metricIdTimers) { this.urlTimers = urlTimers; this.urlAndMethodTimers = urlAndMethodTimers; this.metricIdTimers = metricIdTimers; } public List getUrlTimers() { return urlTimers; } public List getUrlAndMethodTimers() { return urlAndMethodTimers; } public List getMetricIdTimers() { return metricIdTimers; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/MetricIdClientMetricData.java000066400000000000000000000010051312026620400325210ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.puppetlabs.http.client.impl.metrics.TimerMetricData; import java.util.List; public class MetricIdClientMetricData extends ClientMetricData { private final List metricId; public MetricIdClientMetricData(TimerMetricData timerMetricData, List metricId) { super(timerMetricData); this.metricId = metricId; } public List getMetricId() { return metricId; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/MetricIdClientTimer.java000066400000000000000000000011311312026620400315640ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import java.util.List; public class MetricIdClientTimer extends ClientTimer { private final List metricId; public MetricIdClientTimer(String metricName, List metricId, Metrics.MetricType metricType) { super(metricName, metricType); this.metricId = metricId; } public List getMetricId() { return metricId; } @Override public boolean isCategory(Metrics.MetricCategory category) { return category.equals(Metrics.MetricCategory.METRIC_ID); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/Metrics.java000066400000000000000000000237151312026620400273460ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.codahale.metrics.MetricFilter; import com.codahale.metrics.MetricRegistry; import com.codahale.metrics.Timer; import com.puppetlabs.http.client.impl.metrics.CategoryClientTimerMetricFilter; import com.puppetlabs.http.client.impl.metrics.MetricIdClientTimerFilter; import com.puppetlabs.http.client.impl.metrics.TimerMetricData; import com.puppetlabs.http.client.impl.metrics.UrlAndMethodClientTimerFilter; import com.puppetlabs.http.client.impl.metrics.UrlClientTimerFilter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import java.net.URI; import java.net.URISyntaxException; import java.util.ArrayList; import java.util.Arrays; import java.util.List; import java.util.Map; public class Metrics { public static final String PUPPETLABS_NAMESPACE_PREFIX = "puppetlabs"; public static final String HTTP_CLIENT_NAMESPACE_PREFIX = "http-client.experimental"; public static final String DEFAULT_NAMESPACE_PREFIX = PUPPETLABS_NAMESPACE_PREFIX + "." + HTTP_CLIENT_NAMESPACE_PREFIX; public static final String NAMESPACE_URL = "with-url"; public static final String NAMESPACE_URL_AND_METHOD = "with-url-and-method"; public static final String NAMESPACE_METRIC_ID = "with-metric-id"; public static final String NAMESPACE_FULL_RESPONSE = "full-response"; private static final Logger LOGGER = LoggerFactory.getLogger(Metrics.class); public static String buildMetricNamespace(String metricPrefix, String serverId) { if (metricPrefix != null) { if (serverId != null) { Metrics.LOGGER.warn("Metric prefix and server id both set. Using metric prefix '" + metricPrefix + "' for metric namespace."); } return metricPrefix + "." + HTTP_CLIENT_NAMESPACE_PREFIX; } else if (serverId != null) { return PUPPETLABS_NAMESPACE_PREFIX + "." + serverId + "." + HTTP_CLIENT_NAMESPACE_PREFIX; } else { return DEFAULT_NAMESPACE_PREFIX; } } public enum MetricType { FULL_RESPONSE } public enum MetricCategory { URL, URL_AND_METHOD, METRIC_ID } public static String urlToMetricUrl(String uriString) throws URISyntaxException { final URI uri = new URI(uriString); final URI convertedUri = new URI(uri.getScheme(), null, uri.getHost(), uri.getPort(), uri.getPath(), null, null); return convertedUri.toString(); } private static List getUrlClientTimerArray(MetricRegistry registry, MetricFilter filter) { List timerArray = new ArrayList<>(); for (Map.Entry entry : registry.getTimers(filter).entrySet()) { UrlClientTimer timer = (UrlClientTimer)entry.getValue(); timerArray.add(timer); } return timerArray; } private static List getUrlAndMethodClientTimerArray(MetricRegistry registry, MetricFilter filter) { List timerArray = new ArrayList<>(); for (Map.Entry entry : registry.getTimers(filter).entrySet()) { UrlAndMethodClientTimer timer = (UrlAndMethodClientTimer)entry.getValue(); timerArray.add(timer); } return timerArray; } private static List getMetricIdClientTimerArray(MetricRegistry registry, MetricFilter filter) { List timerArray = new ArrayList<>(); for (Map.Entry entry : registry.getTimers(filter).entrySet()) { MetricIdClientTimer timer = (MetricIdClientTimer)entry.getValue(); timerArray.add(timer); } return timerArray; } public static ClientTimerContainer getClientMetrics(MetricRegistry metricRegistry){ if (metricRegistry != null) { return new ClientTimerContainer( getUrlClientTimerArray(metricRegistry, new CategoryClientTimerMetricFilter(MetricCategory.URL)), getUrlAndMethodClientTimerArray(metricRegistry, new CategoryClientTimerMetricFilter(MetricCategory.URL_AND_METHOD)), getMetricIdClientTimerArray(metricRegistry, new CategoryClientTimerMetricFilter(MetricCategory.METRIC_ID))); } else { throw new IllegalArgumentException("Metric registry must not be null"); } } public static List getClientMetricsByUrl(MetricRegistry metricRegistry, final String url){ if (metricRegistry != null) { return getUrlClientTimerArray(metricRegistry, new UrlClientTimerFilter(url)); } else { throw new IllegalArgumentException("Metric registry must not be null"); } } public static List getClientMetricsByUrlAndMethod(MetricRegistry metricRegistry, final String url, final String method){ if (metricRegistry != null) { return getUrlAndMethodClientTimerArray(metricRegistry, new UrlAndMethodClientTimerFilter(url, method)); } else { throw new IllegalArgumentException("Metric registry must not be null"); } } public static List getClientMetricsByMetricId(MetricRegistry metricRegistry, final String[] metricId){ if (metricRegistry != null) { if (metricId.length == 0) { return getMetricIdClientTimerArray(metricRegistry, new CategoryClientTimerMetricFilter(MetricCategory.METRIC_ID)); } else { return getMetricIdClientTimerArray(metricRegistry, new MetricIdClientTimerFilter(new ArrayList(Arrays.asList(metricId)))); } } else { throw new IllegalArgumentException("Metric registry must not be null"); } } private static List computeUrlClientMetricsData(List timers) { if (timers != null) { List metricsData = new ArrayList<>(); for (UrlClientTimer timer: timers) { TimerMetricData timerMetricData = TimerMetricData.fromTimer(timer); String url = timer.getUrl(); metricsData.add(new UrlClientMetricData(timerMetricData, url)); } return metricsData; } else { return null; } } private static List computeUrlAndMethodClientMetricsData(List timers) { if (timers != null) { List metricsData = new ArrayList<>(); for (UrlAndMethodClientTimer timer: timers) { TimerMetricData timerMetricData = TimerMetricData.fromTimer(timer); String url = timer.getUrl(); String method = timer.getMethod(); metricsData.add(new UrlAndMethodClientMetricData(timerMetricData, url, method)); } return metricsData; } else { return null; } } private static List computeMetricIdClientMetricsData(List timers) { if (timers != null) { List metricsData = new ArrayList<>(); for (MetricIdClientTimer timer: timers) { TimerMetricData timerMetricData = TimerMetricData.fromTimer(timer); List metricId = timer.getMetricId(); metricsData.add(new MetricIdClientMetricData(timerMetricData, metricId)); } return metricsData; } else { return null; } } public static ClientMetricDataContainer getClientMetricsData(MetricRegistry metricRegistry){ if ( metricRegistry != null ) { ClientTimerContainer timers = getClientMetrics(metricRegistry); return new ClientMetricDataContainer(computeUrlClientMetricsData(timers.getUrlTimers()), computeUrlAndMethodClientMetricsData(timers.getUrlAndMethodTimers()), computeMetricIdClientMetricsData(timers.getMetricIdTimers()) ); } else { throw new IllegalArgumentException("Metric registry must not be null"); } } public static List getClientMetricsDataByUrl(MetricRegistry metricRegistry, String url){ List timers = getClientMetricsByUrl(metricRegistry, url); return computeUrlClientMetricsData(timers); } public static List getClientMetricsDataByUrlAndMethod(MetricRegistry metricRegistry, String url, String method){ List timers = getClientMetricsByUrlAndMethod(metricRegistry, url, method); return computeUrlAndMethodClientMetricsData(timers); } public static List getClientMetricsDataByMetricId(MetricRegistry metricRegistry, String[] metricId){ List timers = getClientMetricsByMetricId(metricRegistry, metricId); return computeMetricIdClientMetricsData(timers); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/UrlAndMethodClientMetricData.java000066400000000000000000000007571312026620400333640ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.puppetlabs.http.client.impl.metrics.TimerMetricData; public class UrlAndMethodClientMetricData extends UrlClientMetricData { private final String method; public UrlAndMethodClientMetricData(TimerMetricData timerMetricData, String url, String method) { super(timerMetricData, url); this.method = method; } public String getMethod() { return method; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/UrlAndMethodClientTimer.java000066400000000000000000000011171312026620400324160ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; public class UrlAndMethodClientTimer extends UrlClientTimer { private final String method; public UrlAndMethodClientTimer(String metricName, String url, String method, Metrics.MetricType metricType) { super(metricName, url, metricType); this.method = method; } public String getMethod() { return method; } @Override public boolean isCategory(Metrics.MetricCategory category) { return category.equals(Metrics.MetricCategory.URL_AND_METHOD); } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/UrlClientMetricData.java000066400000000000000000000006561312026620400315760ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; import com.puppetlabs.http.client.impl.metrics.TimerMetricData; public class UrlClientMetricData extends ClientMetricData { private final String url; public UrlClientMetricData(TimerMetricData timerMetricData, String url) { super(timerMetricData); this.url = url; } public String getUrl() { return url; } } clj-http-client-0.9.0/src/java/com/puppetlabs/http/client/metrics/UrlClientTimer.java000066400000000000000000000007511312026620400306350ustar00rootroot00000000000000package com.puppetlabs.http.client.metrics; public class UrlClientTimer extends ClientTimer { private final String url; public UrlClientTimer(String metricName, String url, Metrics.MetricType metricType) { super(metricName, metricType); this.url = url; } public String getUrl() { return url; } @Override public boolean isCategory(Metrics.MetricCategory category) { return category.equals(Metrics.MetricCategory.URL); } } clj-http-client-0.9.0/test/000077500000000000000000000000001312026620400154745ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/000077500000000000000000000000001312026620400162525ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/puppetlabs/000077500000000000000000000000001312026620400204315ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/puppetlabs/http/000077500000000000000000000000001312026620400214105ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/puppetlabs/http/client/000077500000000000000000000000001312026620400226665ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/puppetlabs/http/client/impl/000077500000000000000000000000001312026620400236275ustar00rootroot00000000000000clj-http-client-0.9.0/test/com/puppetlabs/http/client/impl/java_client_test.clj000066400000000000000000000047311312026620400276440ustar00rootroot00000000000000(ns com.puppetlabs.http.client.impl.java-client-test (:import (com.puppetlabs.http.client.impl JavaClient) (org.apache.commons.io IOUtils) (com.puppetlabs.http.client ResponseBodyType RequestOptions) (org.apache.http.entity ContentType) (java.io ByteArrayInputStream)) (:require [clojure.test :refer :all])) ;; NOTE: there are more comprehensive, end-to-end tests for ;; the Java client functionality lumped in with the clojure ;; tests. This namespace is just for some Java-only unit tests. (deftest test-coerce-body-type (testing "Can handle a Content Type header with no charset" (let [body "foo" body-stream (IOUtils/toInputStream body "UTF-8")] (is (= "foo" (JavaClient/coerceBodyType body-stream ResponseBodyType/TEXT ContentType/WILDCARD)))))) (defn request-options [body content-type-value] (new RequestOptions nil {"content-type" content-type-value} body false nil)) (defn compute-content-type [body content-type-value] (-> (JavaClient/getContentType body (request-options body content-type-value)) ;; Calling .toString on an instance of org.apache.http.entity.ContentType ;; generates the string that'll actually end up in the header. .toString)) ;; This test case is 100% copypasta from puppetlabs.http.client.async-test (deftest content-type-test (testing "value of content-type header is computed correctly" (testing "a byte stream which specifies application/octet-stream" (let [body (ByteArrayInputStream. (byte-array [(byte 1) (byte 2)]))] (is (= (compute-content-type body "application/octet-stream") "application/octet-stream")))) (testing "the request body is a string" (testing "when a charset is specified, it is honored" (let [body "foo"] (is (= (compute-content-type body "text/plain; charset=US-ASCII") "text/plain; charset=US-ASCII")))) (testing "a missing charset yields a content-type that maintains the given mime-type but adds UTF-8 as the charset" (let [body "foo"] (is (= (compute-content-type body "text/html") "text/html; charset=UTF-8"))))))) (deftest null-response-body-coerced-as-text (testing "a null response body is coerced into a string by JavaClient.coerceBodyType" (let [body nil] (is (= "" (JavaClient/coerceBodyType body ResponseBodyType/TEXT nil)))))) clj-http-client-0.9.0/test/com/puppetlabs/http/client/impl/metrics_unit_test.clj000066400000000000000000000407231312026620400300730ustar00rootroot00000000000000(ns com.puppetlabs.http.client.impl.metrics-unit-test (:require [clojure.test :refer :all] [puppetlabs.http.client.metrics :as metrics] [schema.test :as schema-test]) (:import (com.codahale.metrics MetricRegistry) (com.puppetlabs.http.client.metrics Metrics) (org.apache.http.message BasicHttpRequest) (clojure.lang ExceptionInfo) (com.puppetlabs.http.client.impl.metrics TimerUtils) (java.net URISyntaxException))) (use-fixtures :once schema-test/validate-schemas) (defn add-metric-ns [string] (str "puppetlabs.http-client.experimental." string)) (deftest start-full-response-timers-test (testing "startFullResponseTimers creates the right timers" (let [url-id (add-metric-ns "with-url.http://localhost/foo.full-response") url-method-id (add-metric-ns "with-url-and-method.http://localhost/foo.GET.full-response")] (testing "metric id timers are not created for a request without a metric id" (let [metric-registry (MetricRegistry.)] (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://localhost/foo") nil Metrics/DEFAULT_NAMESPACE_PREFIX true) (is (= (set (list url-id url-method-id)) (set (keys (.getTimers metric-registry))))))) (testing "metric id timers are not created for a request with an empty metric id" (let [metric-registry (MetricRegistry.)] (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://localhost/foo") (into-array String []) Metrics/DEFAULT_NAMESPACE_PREFIX true) (is (= (set (list url-id url-method-id)) (set (keys (.getTimers metric-registry))))))) (testing "metric id timers are created correctly for a request with a metric id" (let [metric-registry (MetricRegistry.)] (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://localhost/foo") (into-array ["foo" "bar" "baz"]) Metrics/DEFAULT_NAMESPACE_PREFIX true) (is (= (set (list url-id url-method-id (add-metric-ns "with-metric-id.foo.full-response") (add-metric-ns "with-metric-id.foo.bar.full-response") (add-metric-ns "with-metric-id.foo.bar.baz.full-response"))) (set (keys (.getTimers metric-registry))))))) (testing "url timers should strip off username, password, query string, and fragment" (let [metric-registry (MetricRegistry.)] (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://user:pwd@localhost:1234/foo%2cbar/baz?te%2cst=one") nil Metrics/DEFAULT_NAMESPACE_PREFIX true) (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://user:pwd@localhost:1234/foo%2cbar/baz#x%2cyz") nil Metrics/DEFAULT_NAMESPACE_PREFIX true) (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://user:pwd@localhost:1234/foo%2cbar/baz?te%2cst=one#x%2cyz") nil Metrics/DEFAULT_NAMESPACE_PREFIX true) (TimerUtils/startFullResponseTimers metric-registry (BasicHttpRequest. "GET" "http://user:pwd@localhost:1234/foo%2cbar/baz?#x%2cyz") nil Metrics/DEFAULT_NAMESPACE_PREFIX true) (is (= (set (list (add-metric-ns "with-url.http://localhost:1234/foo,bar/baz.full-response") (add-metric-ns "with-url-and-method.http://localhost:1234/foo,bar/baz.GET.full-response"))) (set (keys (.getTimers metric-registry)))))))))) (deftest url->metric-url-test (testing "url->metric-url strips username, password, query params, and path fragment off of url" (let [url "http://user:pwd@localhost:1234/foo%2cbar/baz?te%2cst=one"] (is (= "http://localhost:1234/foo,bar/baz" (Metrics/urlToMetricUrl url) (metrics/url->metric-url url))))) (testing "url->metric-url throws error if non-url passed in" (is (thrown? URISyntaxException (Metrics/urlToMetricUrl "abc def"))) (is (thrown? URISyntaxException (metrics/url->metric-url "abc def"))))) (defn start-and-stop-timers! [registry req id] (doseq [timer (TimerUtils/startFullResponseTimers registry req id Metrics/DEFAULT_NAMESPACE_PREFIX true)] (.stop timer))) (deftest get-client-metrics-data-test (let [registry (MetricRegistry.) url "http://test.com/one" url2 "http://test.com/one/two"] (start-and-stop-timers! registry (BasicHttpRequest. "GET" url) nil) (start-and-stop-timers! registry (BasicHttpRequest. "POST" url) nil) (start-and-stop-timers! registry (BasicHttpRequest. "POST" url) (into-array ["foo" "bar"])) (start-and-stop-timers! registry (BasicHttpRequest. "GET" url2) (into-array ["foo" "abc"])) (testing "getClientMetrics without args returns all timers organized by category" (is (= (set [:url :url-and-method :metric-id]) (set (keys (metrics/get-client-metrics registry))) (set (keys (metrics/get-client-metrics-data registry))))) (is (= (set [(add-metric-ns "with-url.http://test.com/one.full-response") (add-metric-ns "with-url.http://test.com/one/two.full-response")]) (set (map #(.getMetricName %) (.getUrlTimers (Metrics/getClientMetrics registry)))) (set (map #(.getMetricName %) (:url (metrics/get-client-metrics registry)))) (set (map #(.getMetricName %) (.getUrlData (Metrics/getClientMetricsData registry)))) (set (map :metric-name (:url (metrics/get-client-metrics-data registry)))))) (is (= (set [(add-metric-ns "with-url-and-method.http://test.com/one.GET.full-response") (add-metric-ns "with-url-and-method.http://test.com/one.POST.full-response") (add-metric-ns "with-url-and-method.http://test.com/one/two.GET.full-response")]) (set (map #(.getMetricName %) (.getUrlAndMethodTimers (Metrics/getClientMetrics registry)))) (set (map #(.getMetricName %) (:url-and-method (metrics/get-client-metrics registry)))) (set (map #(.getMetricName %) (.getUrlAndMethodData (Metrics/getClientMetricsData registry)))) (set (map :metric-name (:url-and-method (metrics/get-client-metrics-data registry)))))) (is (= (set ["puppetlabs.http-client.experimental.with-metric-id.foo.full-response" "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response" "puppetlabs.http-client.experimental.with-metric-id.foo.abc.full-response"]) (set (map #(.getMetricName %) (.getMetricIdTimers (Metrics/getClientMetrics registry)))) (set (map #(.getMetricName %) (:metric-id (metrics/get-client-metrics registry)))) (set (map #(.getMetricName %) (.getMetricIdData (Metrics/getClientMetricsData registry)))) (set (map :metric-name (:metric-id (metrics/get-client-metrics-data registry))))))) (testing "getClientMetricsData with url returns the right thing" (let [java-data (Metrics/getClientMetricsDataByUrl registry url) clj-data (metrics/get-client-metrics-data-by-url registry url)] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-url.http://test.com/one.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 3 (.getCount (first java-data)) (:count (first clj-data))))) (let [java-data (Metrics/getClientMetricsDataByUrl registry url2) clj-data (metrics/get-client-metrics-data-by-url registry url2)] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-url.http://test.com/one/two.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 1 (.getCount (first java-data)) (:count (first clj-data))))) (testing "getClientMetricsData with url returns nothing if url is not a full match" (is (= [] (Metrics/getClientMetricsDataByUrl registry "http://test.com") (metrics/get-client-metrics-data-by-url registry "http://test.com"))))) (testing "getClientMetricsData with url and method returns the right thing" (let [java-data (Metrics/getClientMetricsDataByUrlAndMethod registry url "GET") clj-data (metrics/get-client-metrics-data-by-url-and-method registry url :get)] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-url-and-method.http://test.com/one.GET.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 1 (.getCount (first java-data)) (:count (first clj-data))))) (let [java-data (Metrics/getClientMetricsDataByUrlAndMethod registry url "POST") clj-data (metrics/get-client-metrics-data-by-url-and-method registry url :post)] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-url-and-method.http://test.com/one.POST.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 2 (.getCount (first java-data)) (:count (first clj-data))))) (let [java-data (Metrics/getClientMetricsDataByUrlAndMethod registry url2 "GET") clj-data (metrics/get-client-metrics-data-by-url-and-method registry url2 :get)] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-url-and-method.http://test.com/one/two.GET.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 1 (.getCount (first java-data)) (:count (first clj-data))))) (testing "getClientMetricsData with url and method returns nothing if method is not a match" (is (= [] (Metrics/getClientMetricsDataByUrlAndMethod registry "http://test.com" "PUT") (metrics/get-client-metrics-data-by-url-and-method registry "http://test.com" :put))))) (testing "getClientMetricsData with metric id returns the right thing" (let [java-data (Metrics/getClientMetricsDataByMetricId registry (into-array ["foo"])) clj-data (metrics/get-client-metrics-data-by-metric-id registry ["foo"])] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-metric-id.foo.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 2 (.getCount (first java-data)) (:count (first clj-data))))) (let [java-data (Metrics/getClientMetricsDataByMetricId registry (into-array ["foo" "bar"])) clj-data (metrics/get-client-metrics-data-by-metric-id registry ["foo" "bar"])] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-metric-id.foo.bar.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 1 (.getCount (first java-data)) (:count (first clj-data))))) (let [java-data (Metrics/getClientMetricsDataByMetricId registry (into-array ["foo" "abc"])) clj-data (metrics/get-client-metrics-data-by-metric-id registry ["foo" "abc"])] (is (= 1 (count java-data) (count clj-data))) (is (= (add-metric-ns "with-metric-id.foo.abc.full-response") (.getMetricName (first java-data)) (:metric-name (first clj-data)))) (is (= 1 (.getCount (first java-data)) (:count (first clj-data)))) (testing "metric id can be specified as keyword or string" (is (= clj-data (metrics/get-client-metrics-data-by-metric-id registry ["foo" :abc]))))) (testing "getClientMetricsData with metric id returns nothing if id is not a match" (is (= [] (Metrics/getClientMetricsDataByMetricId registry (into-array ["foo" "cat"])) (metrics/get-client-metrics-data-by-metric-id registry ["foo" "cat"])))) (testing "getClientMetrics|Data returns throws an error if no metric registry passed in" (is (thrown? ExceptionInfo (metrics/get-client-metrics nil))) (is (thrown? ExceptionInfo (metrics/get-client-metrics-data nil))) (is (thrown? IllegalArgumentException (Metrics/getClientMetrics nil))) (is (thrown? IllegalArgumentException (Metrics/getClientMetricsData nil)))) (testing (str "getClientMetrics|Data returns data structure with empty arrays" " as values if no requests have been made yet") (let [empty-metrics (Metrics/getClientMetrics (MetricRegistry.)) empty-metrics-data (Metrics/getClientMetricsData (MetricRegistry.))] (is (= {:url [] :url-and-method [] :metric-id []} (metrics/get-client-metrics (MetricRegistry.)) (metrics/get-client-metrics-data (MetricRegistry.)))) (is (empty? (.getUrlTimers empty-metrics))) (is (empty? (.getUrlAndMethodTimers empty-metrics))) (is (empty? (.getMetricIdTimers empty-metrics))) (is (empty? (.getUrlData empty-metrics-data))) (is (empty? (.getUrlAndMethodData empty-metrics-data))) (is (empty? (.getMetricIdData empty-metrics-data))))) (testing "getClientMetrics returns correctly without metric-id on request" (let [registry (MetricRegistry.) url "http://test.com/one"] (start-and-stop-timers! registry (BasicHttpRequest. "GET" url) nil) (let [client-metrics (Metrics/getClientMetrics registry) client-metrics-data (Metrics/getClientMetricsData registry)] (is (= (set [(add-metric-ns "with-url.http://test.com/one.full-response")]) (set (map #(.getMetricName %) (.getUrlTimers client-metrics))) (set (map #(.getMetricName %) (.getUrlData client-metrics-data))))) (is (= (set [(add-metric-ns "with-url-and-method.http://test.com/one.GET.full-response")]) (set (map #(.getMetricName %) (.getUrlAndMethodTimers client-metrics))) (set (map #(.getMetricName %) (.getUrlAndMethodData client-metrics-data))))) (is (= [] (.getMetricIdTimers client-metrics) (.getMetricIdData client-metrics-data))))))))) (deftest empty-metric-id-filter-test (testing "a metric id filter with an empty array returns all metric id timers" (let [registry (MetricRegistry.) url "http://test.com/foo/bar" foo-id (add-metric-ns "with-metric-id.foo.full-response") foo-bar-id (add-metric-ns "with-metric-id.foo.bar.full-response") foo-bar-baz-id (add-metric-ns "with-metric-id.foo.bar.baz.full-response")] (start-and-stop-timers! registry (BasicHttpRequest. "GET" url) (into-array ["foo" "bar" "baz"])) (testing "empty metric filter returns all metric id timers" (is (= (set (list foo-id foo-bar-id foo-bar-baz-id)) (set (map #(.getMetricName %) (Metrics/getClientMetricsDataByMetricId registry (into-array String [])))) (set (map :metric-name (metrics/get-client-metrics-data-by-metric-id registry []))))))))) clj-http-client-0.9.0/test/puppetlabs/000077500000000000000000000000001312026620400176535ustar00rootroot00000000000000clj-http-client-0.9.0/test/puppetlabs/http/000077500000000000000000000000001312026620400206325ustar00rootroot00000000000000clj-http-client-0.9.0/test/puppetlabs/http/client/000077500000000000000000000000001312026620400221105ustar00rootroot00000000000000clj-http-client-0.9.0/test/puppetlabs/http/client/async_plaintext_test.clj000066400000000000000000000557311312026620400270610ustar00rootroot00000000000000(ns puppetlabs.http.client.async-plaintext-test (:import (com.puppetlabs.http.client Async RequestOptions ClientOptions) (org.apache.http.impl.nio.client HttpAsyncClients) (java.net URI SocketTimeoutException ServerSocket) (java.util Locale)) (:require [clojure.test :refer :all] [puppetlabs.http.client.test-common :refer :all] [puppetlabs.i18n.core :as i18n] [puppetlabs.trapperkeeper.core :as tk] [puppetlabs.trapperkeeper.testutils.bootstrap :as testutils] [puppetlabs.trapperkeeper.testutils.logging :as testlogging] [puppetlabs.trapperkeeper.testutils.webserver :as testwebserver] [puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9] [puppetlabs.http.client.common :as common] [puppetlabs.http.client.async :as async] [schema.test :as schema-test] [ring.middleware.cookies :refer [wrap-cookies]])) (use-fixtures :once schema-test/validate-schemas) (defn app [_] {:status 200 :body "Hello, World!"}) (defn app-with-empty-content-type [_] {:headers {"content-type" ""} :status 200 :body "Hello, World!"}) (defn app-with-language-header-echo [{{:strs [accept-language]} :headers}] {:status 200 :body (str accept-language)}) (tk/defservice test-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler app "/hello") context)) (defn cookie-handler [_] {:status 200 :body "cookie has been set" :cookies {"session_id" {:value "session-id-hash"}}}) (defn check-cookie-handler [req] (if (empty? (get req :cookies)) {:status 400 :body "cookie has not been set"} {:status 200 :body "cookie has been set"})) (tk/defservice test-cookie-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler (wrap-cookies cookie-handler) "/cookietest") (add-ring-handler (wrap-cookies check-cookie-handler) "/cookiecheck") context)) (deftest persistent-async-client-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (testing "java async client" (let [request-options (RequestOptions. (URI. "http://localhost:10000/hello/")) client-options (ClientOptions.) client (Async/createClient client-options)] (testing "HEAD request with persistent async client" (let [response (.head client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= nil (.getBody (.deref response)))))) (testing "GET request with persistent async client" (let [response (.get client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "POST request with persistent async client" (let [response (.post client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "PUT request with persistent async client" (let [response (.put client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "DELETE request with persistent async client" (let [response (.delete client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "TRACE request with persistent async client" (let [response (.trace client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "OPTIONS request with persistent async client" (let [response (.options client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "PATCH request with persistent async client" (let [response (.patch client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response))))))) (testing "client closes properly" (.close client) (is (thrown? IllegalStateException (.get client request-options)))))) (testing "clojure async client" (let [client (async/create-client {})] (testing "HEAD request with persistent async client" (let [response (common/head client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= nil (:body @response))))) (testing "GET request with persistent async client" (let [response (common/get client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "POST request with persistent async client" (let [response (common/post client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "PUT request with persistent async client" (let [response (common/put client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "DELETE request with persistent async client" (let [response (common/delete client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "TRACE request with persistent async client" (let [response (common/trace client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "OPTIONS request with persistent async client" (let [response (common/options client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "PATCH request with persistent async client" (let [response (common/patch client "http://localhost:10000/hello/")] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "GET request via request function with persistent async client" (let [response (common/make-request client "http://localhost:10000/hello/" :get)] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "Bad verb request via request function with persistent async client" (is (thrown? IllegalArgumentException (common/make-request client "http://localhost:10000/hello/" :bad)))) (testing "client closes properly" (common/close client) (is (thrown? IllegalStateException (common/get client "http://localhost:10000/hello/"))))))))) (deftest java-api-cookie-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-cookie-service] {:webserver {:port 10000}} (let [client (Async/createClient (ClientOptions.))] (testing "Set a cookie using Java API" (let [response (.get client (RequestOptions. (URI. "http://localhost:10000/cookietest")))] (is (= 200 (.getStatus (.deref response)))))) (testing "Check if cookie still exists" (let [response (.get client (RequestOptions. (URI. "http://localhost:10000/cookiecheck")))] (is (= 200 (.getStatus (.deref response)))))))))) (deftest clj-api-cookie-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-cookie-service] {:webserver {:port 10000}} (let [client (async/create-client {})] (testing "Set a cookie using Clojure API" (let [response (common/get client "http://localhost:10000/cookietest")] (is (= 200 (:status @response))))) (testing "Check if cookie still exists" (let [response (common/get client "http://localhost:10000/cookiecheck")] (is (= 200 (:status @response))))))))) (deftest request-with-client-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (let [client (HttpAsyncClients/createDefault) opts {:method :get :url "http://localhost:10000/hello/"}] (.start client) (testing "GET request works with request-with-client" (let [response (async/request-with-client opts nil client)] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (testing "Client persists when passed to request-with-client" (let [response (async/request-with-client opts nil client)] (is (= 200 (:status @response))) (is (= "Hello, World!" (slurp (:body @response)))))) (.close client))))) (deftest query-params-test-async (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-params-web-service] {:webserver {:port 8080}} (testing "URL Query Parameters work with the Java client" (let [client (Async/createClient (ClientOptions.))] (try (let [request-options (RequestOptions. (URI. "http://localhost:8080/params?foo=bar&baz=lux")) response (.get client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= queryparams (read-string (slurp (.getBody (.deref response))))))) (finally (.close client))))) (testing "URL Query Parameters work with the clojure client" (with-open [client (async/create-client {})] (let [opts {:method :get :url "http://localhost:8080/params/" :query-params queryparams :as :text} response (common/get client "http://localhost:8080/params" opts)] (is (= 200 (:status @response))) (is (= queryparams (read-string (:body @response))))))) (testing "URL Query Parameters can be set directly in the URL" (with-open [client (async/create-client {})] (let [response (common/get client "http://localhost:8080/params?paramone=one" {:as :text})] (is (= 200 (:status @response))) (is (= (str {"paramone" "one"}) (:body @response)))))) (testing (str "URL Query Parameters set in URL are overwritten if params " "are also specified in options map") (with-open [client (async/create-client {})] (let [response (common/get client "http://localhost:8080/params?paramone=one&foo=lux" query-options)] (is (= 200 (:status @response))) (is (= queryparams (read-string (:body @response)))))))))) (deftest redirect-test-async (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service redirect-web-service] {:webserver {:port 8080}} (testing (str "redirects on POST not followed by persistent Java client " "when forceRedirects option not set to true") (let [client (Async/createClient (ClientOptions.))] (try (let [request-options (RequestOptions. (URI. "http://localhost:8080/hello")) response (.post client request-options)] (is (= 302 (.getStatus (.deref response))))) (finally (.close client))))) (testing "redirects on POST followed by Java client when option is set" (let [client (Async/createClient (.. (ClientOptions.) (setForceRedirects true)))] (try (let [request-options (RequestOptions. (URI. "http://localhost:8080/hello")) response (.post client request-options)] (is (= 200 (.getStatus (.deref response)))) (is (= "Hello, World!" (slurp (.getBody (.deref response)))))) (finally (.close client))))) (testing "redirects not followed by Java client when :follow-redirects is false" (let [client (Async/createClient (.. (ClientOptions.) (setFollowRedirects false)))] (try (let [request-options (RequestOptions. (URI. "http://localhost:8080/hello")) response (.get client request-options)] (is (= 302 (.getStatus (.deref response))))) (finally (.close client))))) (testing ":follow-redirects overrides :force-redirects for Java client" (let [client (Async/createClient (.. (ClientOptions.) (setFollowRedirects false) (setForceRedirects true)))] (try (let [request-options (RequestOptions. (URI. "http://localhost:8080/hello")) response (.get client request-options)] (is (= 302 (.getStatus (.deref response))))) (finally (.close client))))) (testing (str "redirects on POST not followed by clojure client " "when :force-redirects is not set to true") (with-open [client (async/create-client {:force-redirects false})] (let [opts {:method :post :url "http://localhost:8080/hello" :as :text} response (common/post client "http://localhost:8080/hello" opts)] (is (= 302 (:status @response)))))) (testing (str "redirects on POST followed by persistent clojure client " "when option is set") (with-open [client (async/create-client {:force-redirects true})] (let [response (common/post client "http://localhost:8080/hello" {:as :text})] (is (= 200 (:status @response))) (is (= "Hello, World!" (:body @response)))))) (testing (str "persistent clojure client does not follow redirects when " ":follow-redirects is set to false") (with-open [client (async/create-client {:follow-redirects false})] (let [response (common/get client "http://localhost:8080/hello" {:as :text})] (is (= 302 (:status @response)))))) (testing ":follow-redirects overrides :force-redirects with persistent clj client" (with-open [client (async/create-client {:follow-redirects false :force-redirects true})] (let [response (common/get client "http://localhost:8080/hello" {:as :text})] (is (= 302 (:status @response))))))))) (deftest short-connect-timeout-persistent-java-test-async (testing (str "connection times out properly for java persistent client " "async request with short timeout") (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 250) (Async/createClient))] (let [request-options (RequestOptions. "http://127.0.0.255:65535") time-before-connect (System/currentTimeMillis)] (is (connect-exception-thrown? (-> client (.get request-options) (.deref) (.getError))) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout"))))) (deftest short-connect-timeout-persistent-clojure-test-async (testing (str "connection times out properly for clojure persistent client " "async request with short timeout") (with-open [client (async/create-client {:connect-timeout-milliseconds 250})] (let [time-before-connect (System/currentTimeMillis)] (is (connect-exception-thrown? (-> @(common/get client "http://127.0.0.255:65535") :error)) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout"))))) (deftest longer-connect-timeout-test-async (testing "connection succeeds for async request with longer connect timeout" (testlogging/with-test-logging (testwebserver/with-test-webserver app port (let [url (str "http://localhost:" port "/hello")] (testing "java persistent async client" (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 2000) (Async/createClient))] (let [response (-> client (.get (RequestOptions. url)) (.deref))] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response))))))) (testing "clojure persistent async client" (with-open [client (async/create-client {:connect-timeout-milliseconds 2000})] (let [response @(common/get client url {:as :text})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))))))))) (deftest short-socket-timeout-persistent-java-test-async (testing (str "socket read times out properly for persistent java async " "request with short timeout") (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 1) (Async/createClient)) server (ServerSocket. 0)] (let [request-options (-> "http://127.0.0.1:" (str (.getLocalPort server)) (RequestOptions.)) time-before-connect (System/currentTimeMillis)] (is (instance? SocketTimeoutException (-> client (.get request-options) (.deref) (.getError))) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest short-socket-timeout-persistent-clojure-test-async (testing (str "socket read times out properly for clojure persistent client " "async request with short timeout") (with-open [client (async/create-client {:socket-timeout-milliseconds 250}) server (ServerSocket. 0)] (let [url (str "http://127.0.0.1:" (.getLocalPort server)) time-before-connect (System/currentTimeMillis)] (is (instance? SocketTimeoutException (-> @(common/get client url) :error)) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest longer-socket-timeout-test-async (testing "get succeeds for async request with longer socket timeout" (testlogging/with-test-logging (testwebserver/with-test-webserver app port (let [url (str "http://localhost:" port "/hello")] (testing "java persistent async client" (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 2000) (Async/createClient))] (let [response (-> client (.get (RequestOptions. url)) (.deref))] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response))))))) (testing "clojure persistent async client" (with-open [client (async/create-client {:socket-timeout-milliseconds 2000})] (let [response @(common/get client url {:as :text})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))))))))) (deftest empty-content-type-async (testing "content-type parsing handles empty content-type" (testlogging/with-test-logging (testwebserver/with-test-webserver app-with-empty-content-type port (let [url (str "http://localhost:" port "/hello")] (testing "java persistent async client" (with-open [client (-> (ClientOptions.) (Async/createClient))] (let [response (-> client (.get (RequestOptions. url)) (.deref))] (is (= 200 (.getStatus response)))))) (testing "clojure persistent async client" (with-open [client (async/create-client {})] (let [response @(common/get client url {:as :text})] (is (= 200 (:status response))))))))))) (deftest accept-language-async (testing "client passes on the user-locale in Accept-Language header" (testlogging/with-test-logging (testwebserver/with-test-webserver app-with-language-header-echo port (i18n/with-user-locale (Locale. "es" "ES") (let [url (str "http://localhost:" port "/hello")] (testing "clojure persistent async client" (with-open [client (async/create-client {})] (let [response @(common/get client url {:as :text})] (is (= 200 (:status response))) (is (= "es-ES" (:body response)))))))))))) clj-http-client-0.9.0/test/puppetlabs/http/client/async_unbuffered_test.clj000066400000000000000000000420511312026620400271650ustar00rootroot00000000000000(ns puppetlabs.http.client.async-unbuffered-test (:import (com.puppetlabs.http.client Async RequestOptions ClientOptions ResponseBodyType) (java.net SocketTimeoutException ConnectException) (java.io PipedInputStream PipedOutputStream) (java.util.concurrent TimeoutException) (java.util UUID)) (:require [clojure.test :refer :all] [puppetlabs.http.client.test-common :refer :all] [puppetlabs.trapperkeeper.testutils.logging :as testlogging] [puppetlabs.trapperkeeper.testutils.webserver :as testwebserver] [puppetlabs.http.client.common :as common] [puppetlabs.http.client.async :as async] [schema.test :as schema-test])) (use-fixtures :once schema-test/validate-schemas) (defn generate-data "Generate data of approximately the requested size, which is moderately compressible" [data-size] (apply str "xxxx" (repeatedly (/ data-size 35) #(UUID/randomUUID)))) (defn successful-handler "A Ring handler that asynchronously sends some data, waits for confirmation the data has been received then sends some more data" [data send-more-data] (fn [_] (let [outstream (PipedOutputStream.) instream (PipedInputStream.)] (.connect instream outstream) ;; Return the response immediately and asynchronously stream some data into it (future (.write outstream (.getBytes data)) ; Block until the client confirms it has read the first few bytes ; :socket-timeout-milliseconds on the client ensures we can't really get stuck here, even if the test fails (if send-more-data (deref send-more-data)) ; Write the last of the data (.write outstream (.getBytes "yyyy")) (.close outstream)) {:status 200 :body instream}))) (defn blocking-handler "A Ring handler that sends some data but then never closes the socket" [data] (fn [_] (let [outstream (PipedOutputStream.) instream (PipedInputStream.)] (.connect instream outstream) ;; Return the response immediately and asynchronously stream some data into it (future (.write outstream (.getBytes data))) {:status 200 :body instream}))) (defn- clojure-non-blocking-streaming "Stream 32M of data (roughly) which is large enough to ensure the client won't buffer it all. Checks the data is streamed in a non-blocking manner i.e some data is received by the client before the server has finished transmission" [decompress-body?] (testlogging/with-test-logging (let [data (generate-data (* 32 1024 1024)) opts {:as :unbuffered-stream :decompress-body decompress-body?}] (testing " - check data can be streamed successfully" (let [send-more-data (promise)] (testwebserver/with-test-webserver-and-config (successful-handler data send-more-data) port {:shutdown-timeout-seconds 1} (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 20000})] (let [response @(common/get client (str "http://localhost:" port "/hello") opts) {:keys [status body]} response] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4) _ (.read instream buf)] (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (deliver send-more-data true) ;; Indicate we read some chars (is (= (str data "yyyy") (str "xxxx" (slurp instream)))))))))) ;; Read the rest and validate (testing " - check socket timeout is handled" (try (testwebserver/with-test-webserver-and-config (blocking-handler data) port {:shutdown-timeout-seconds 1} (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 200})] (let [response @(common/get client (str "http://localhost:" port "/hello") opts) {:keys [body error]} response] (is (nil? error)) ;; Consume the body to get the exception (is (thrown? SocketTimeoutException (slurp body)))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated ))) (testing " - check connection timeout is handled" (with-open [client (async/create-client {:connect-timeout-milliseconds 100})] (let [response @(common/get client (str "http://localhost:" 12345 "/bad") opts) {:keys [error]} response] (is error) (is (instance? ConnectException error)))))))) (deftest clojure-non-blocking-streaming-without-decompression (testing "clojure :unbuffered-stream with 32MB payload and no decompression" (clojure-non-blocking-streaming false))) (deftest clojure-non-blocking-streaming-with-decompression (testing "clojure :unbuffered-stream with 32MB payload and decompression" (clojure-non-blocking-streaming true))) (defn- clojure-blocking-streaming "Stream data that is buffered client-side i.e. in a blocking manner" [data opts] (testlogging/with-test-logging (testing " - check data can be streamed successfully" (testwebserver/with-test-webserver-and-config (successful-handler data nil) port {:shutdown-timeout-seconds 1} (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 20000})] (let [response @(common/get client (str "http://localhost:" port "/hello") opts) {:keys [status body]} response] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4) _ (.read instream buf)] (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (is (= (str data "yyyy") (str "xxxx" (slurp instream))))))))) ;; Read the rest and validate (testing " - check socket timeout is handled" (try (testwebserver/with-test-webserver-and-config (blocking-handler data) port {:shutdown-timeout-seconds 1} (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 200})] (let [response @(common/get client (str "http://localhost:" port "/hello") opts) {:keys [error]} response] (is (instance? SocketTimeoutException error))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated ))) (testing " - check connection timeout is handled" (with-open [client (async/create-client {:connect-timeout-milliseconds 100})] (let [response @(common/get client (str "http://localhost:" 12345 "/bad") opts) {:keys [error]} response] (is error) (is (instance? ConnectException error))))))) (deftest clojure-blocking-streaming-without-decompression (testing "clojure :unbuffered-stream with 1K payload and no decompression" ;; This is a small enough payload that :unbuffered-stream still buffers it all in memory and so it behaves ;; identically to :stream (clojure-blocking-streaming (generate-data 1024) {:as :unbuffered-stream :decompress-body false}))) (deftest clojure-blocking-streaming-with-decompression (testing "clojure :unbuffered-stream with 1K payload and decompression" ;; This is a small enough payload that :unbuffered-stream still buffers it all in memory and so it behaves ;; identically to :stream (clojure-blocking-streaming (generate-data 1024) {:as :unbuffered-stream :decompress-body true}))) (deftest clojure-existing-streaming-with-small-payload-without-decompression (testing "clojure :stream with 1K payload and no decompression" (clojure-blocking-streaming (generate-data 1024) {:as :stream :decompress-body false}))) (deftest clojure-existing-streaming-with-small-payload-with-decompression (testing "clojure :stream with 1K payload and decompression" (clojure-blocking-streaming (generate-data 1024) {:as :stream :decompress-body true}))) (deftest clojure-existing-streaming-with-large-payload-without-decompression (testing "clojure :stream with 32M payload and no decompression" (clojure-blocking-streaming (generate-data (* 32 1024 1024)) {:as :stream :decompress-body false}))) (deftest clojure-existing-streaming-with-large-payload-with-decompression (testing "clojure :stream with 32M payload and decompression" (clojure-blocking-streaming (generate-data (* 32 1024 1024)) {:as :stream :decompress-body true}))) (defn- java-non-blocking-streaming "Stream 32M of data (roughly) which is large enough to ensure the client won't buffer it all. Checks the data is streamed in a non-blocking manner i.e some data is received by the client before the server has finished transmission" [decompress-body?] (testlogging/with-test-logging (let [data (generate-data (* 32 1024 1024))] (testing " - check data can be streamed successfully" (let [send-more-data (promise)] (testwebserver/with-test-webserver-and-config (successful-handler data send-more-data) port {:shutdown-timeout-seconds 1} (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 20000) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" port "/hello")) (.setAs ResponseBodyType/UNBUFFERED_STREAM) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) status (.getStatus response) body (.getBody response)] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4) _ (.read instream buf)] (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (deliver send-more-data true) ;; Indicate we read some chars (is (= (str data "yyyy") (str "xxxx" (slurp instream)))))))))) ;; Read the rest and validate (testing " - check socket timeout is handled" (try (testwebserver/with-test-webserver-and-config (blocking-handler data) port {:shutdown-timeout-seconds 1} (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 200) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" port "/hello")) (.setAs ResponseBodyType/UNBUFFERED_STREAM) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) body (.getBody response) error (.getError response)] (is (nil? error)) ;; Consume the body to get the exception (is (thrown? SocketTimeoutException (slurp body)))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated ))) (testing " - check connection timeout is handled" (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" 12345 "/bad")) (.setAs ResponseBodyType/UNBUFFERED_STREAM) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) error (.getError response)] (is error) (is (instance? ConnectException error)))))))) (deftest java-non-blocking-streaming-without-decompression (testing "java :unbuffered-stream with 32MB payload and no decompression" (java-non-blocking-streaming false))) (deftest java-non-blocking-streaming-with-decompression (testing "java :unbuffered-stream with 32MB payload and decompression" (java-non-blocking-streaming true))) (defn- java-blocking-streaming "Stream data that is buffered client-side i.e. in a blocking manner" [data response-body-type decompress-body?] (testlogging/with-test-logging (testing " - check data can be streamed successfully" (testwebserver/with-test-webserver-and-config (successful-handler data nil) port {:shutdown-timeout-seconds 1} (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 20000) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" port "/hello")) (.setAs response-body-type) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) status (.getStatus response) body (.getBody response)] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4) _ (.read instream buf)] (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (is (= (str data "yyyy") (str "xxxx" (slurp instream))))))))) ;; Read the rest and validate (testing " - check socket timeout is handled" (try (testwebserver/with-test-webserver-and-config (blocking-handler data) port {:shutdown-timeout-seconds 1} (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 200) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" port "/hello")) (.setAs response-body-type) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) error (.getError response)] (is (instance? SocketTimeoutException error))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated ))) (testing " - check connection timeout is handled" (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 100) (Async/createClient))] (let [request-options (doto (RequestOptions. (str "http://localhost:" 12345 "/bad")) (.setAs response-body-type) (.setDecompressBody decompress-body?)) response (-> client (.get request-options) .deref) error (.getError response)] (is error) (is (instance? ConnectException error))))))) (deftest java-blocking-streaming-without-decompression (testing "java :unbuffered-stream with 1K payload and no decompression" ;; This is a small enough payload that :unbuffered-stream still buffers it all in memory and so it behaves ;; identically to :stream (java-blocking-streaming (generate-data 1024) ResponseBodyType/UNBUFFERED_STREAM false))) (deftest java-blocking-streaming-with-decompression (testing "java :unbuffered-stream with 1K payload and decompression" ;; This is a small enough payload that :unbuffered-stream still buffers it all in memory and so it behaves ;; identically to :stream (java-blocking-streaming (generate-data 1024) ResponseBodyType/UNBUFFERED_STREAM true))) (deftest java-existing-streaming-with-small-payload-without-decompression (testing "java :stream with 1K payload and no decompression" (java-blocking-streaming (generate-data 1024) ResponseBodyType/STREAM false))) (deftest java-existing-streaming-with-small-payload-with-decompression (testing "java :stream with 1K payload and decompression" (java-blocking-streaming (generate-data 1024) ResponseBodyType/STREAM true))) (deftest java-existing-streaming-with-large-payload-without-decompression (testing "java :stream with 32M payload and no decompression" (java-blocking-streaming (generate-data (* 32 1024 1024)) ResponseBodyType/STREAM false))) (deftest java-existing-streaming-with-large-payload-with-decompression (testing "java :stream with 32M payload and decompression" (java-blocking-streaming (generate-data (* 32 1024 1024)) ResponseBodyType/STREAM true))) clj-http-client-0.9.0/test/puppetlabs/http/client/gzip_request_test.clj000066400000000000000000000143661312026620400263740ustar00rootroot00000000000000(ns puppetlabs.http.client.gzip-request-test (:import (com.puppetlabs.http.client Sync SimpleRequestOptions ResponseBodyType CompressType) (java.io ByteArrayInputStream FilterInputStream) (java.net URI) (java.util.zip GZIPInputStream)) (:require [clojure.test :refer :all] [cheshire.core :as cheshire] [schema.test :as schema-test] [puppetlabs.http.client.sync :as http-client] [puppetlabs.http.client.test-common :refer :all] [puppetlabs.trapperkeeper.testutils.webserver :as testwebserver])) (use-fixtures :once schema-test/validate-schemas) (defn req-body-app [req] (let [response {:request-content-encoding (get-in req [:headers "content-encoding"]) :request-body-decompressed (slurp (GZIPInputStream. (:body req)) :encoding "utf-8")}] {:status 200 :headers {"Content-Type" "application/json; charset=utf-8"} :body (cheshire/generate-string response)})) (def short-request-body "gzip me�") (def big-request-body (apply str (repeat 4000 "and�i�said�hey�yeah�yeah�whats�going�on"))) (defn string->byte-array-input-stream [source is-closed-atom] (let [bis (-> source (.getBytes) (ByteArrayInputStream.))] (proxy [FilterInputStream] [bis] (close [] (reset! is-closed-atom true) (proxy-super close))))) (defn post-gzip-clj-request [port body] (-> (http-client/post (format "http://localhost:%d" port) {:body body :headers {"Content-Type" "text/plain; charset=utf-8"} :compress-request-body :gzip :as :text}) :body (cheshire/parse-string true))) (defn post-gzip-java-request [port body] (-> (SimpleRequestOptions. (URI. (format "http://localhost:%d/hello/" port))) (.setBody body) (.setHeaders {"Content-Type" "text/plain; charset=utf-8"}) (.setRequestBodyCompression CompressType/GZIP) (.setAs ResponseBodyType/TEXT) (Sync/post) (.getBody) (cheshire/parse-string true))) (deftest clj-sync-client-gzip-requests (testing "for clojure sync client" (testwebserver/with-test-webserver req-body-app port (testing "short string body is gzipped in request" (let [response (post-gzip-clj-request port short-request-body)] (is (= "gzip" (:request-content-encoding response))) (is (= short-request-body (:request-body-decompressed response))))) (testing "big string body is gzipped in request" (let [response (post-gzip-clj-request port big-request-body)] (is (= "gzip" (:request-content-encoding response))) (is (= big-request-body (:request-body-decompressed response))))) (testing "short inputstream body is gzipped in request" (let [is-closed (atom false) response (post-gzip-clj-request port (string->byte-array-input-stream short-request-body is-closed))] (is (= "gzip" (:request-content-encoding response))) (is (= short-request-body (:request-body-decompressed response))) (is @is-closed "input stream was not closed after request"))) (testing "big inputstream body is gzipped in request" (let [is-closed (atom false) response (post-gzip-clj-request port (string->byte-array-input-stream big-request-body is-closed))] (is (= "gzip" (:request-content-encoding response))) (is (= big-request-body (:request-body-decompressed response))) (is @is-closed "input stream was not closed after request")))))) (deftest java-sync-client-gzip-requests (testing "for java sync client" (testwebserver/with-test-webserver req-body-app port (testing "short string body is gzipped in request" (let [response (post-gzip-java-request port short-request-body)] (is (= "gzip" (:request-content-encoding response))) (is (= short-request-body (:request-body-decompressed response))))) (testing "big string body is gzipped in request" (let [response (post-gzip-java-request port big-request-body)] (is (= "gzip" (:request-content-encoding response))) (is (= big-request-body (:request-body-decompressed response))))) (testing "short inputstream body is gzipped in request" (let [is-closed (atom false) response (post-gzip-java-request port (string->byte-array-input-stream short-request-body is-closed))] (is (= "gzip" (:request-content-encoding response))) (is (= short-request-body (:request-body-decompressed response))) (is @is-closed "input stream was not closed after request"))) (testing "big inputstream body is gzipped in request" (let [is-closed (atom false) response (post-gzip-java-request port (string->byte-array-input-stream big-request-body is-closed))] (is (= "gzip" (:request-content-encoding response))) (is (= big-request-body (:request-body-decompressed response))) (is @is-closed "input stream was not closed after request")))))) (deftest connect-exception-during-gzip-request-returns-failure (testing "connection exception during gzip request returns failure" (let [is-closed (atom false)] (is (connect-exception-thrown? (http-client/post "http://localhost:65535" {:body (string->byte-array-input-stream short-request-body is-closed) :compress-request-body :gzip :as :text}))) (is @is-closed "input stream was not closed after request")))) clj-http-client-0.9.0/test/puppetlabs/http/client/metrics_test.clj000066400000000000000000001424461312026620400253220ustar00rootroot00000000000000(ns puppetlabs.http.client.metrics-test (:require [clojure.test :refer :all] [puppetlabs.http.client.async-unbuffered-test :as unbuffered-test] [puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9] [puppetlabs.trapperkeeper.testutils.bootstrap :as testutils] [puppetlabs.trapperkeeper.testutils.logging :as testlogging] [puppetlabs.trapperkeeper.testutils.webserver :as testwebserver] [puppetlabs.http.client.async :as async] [puppetlabs.http.client.sync :as sync] [puppetlabs.http.client.common :as common] [puppetlabs.trapperkeeper.core :as tk] [puppetlabs.http.client.metrics :as metrics] [schema.test :as schema-test]) (:import (com.puppetlabs.http.client Async RequestOptions ClientOptions ResponseBodyType Sync) (com.codahale.metrics MetricRegistry) (java.net SocketTimeoutException) (java.util.concurrent TimeoutException) (com.puppetlabs.http.client.metrics Metrics ClientTimer ClientMetricData))) (use-fixtures :once schema-test/validate-schemas) (def metric-namespace "puppetlabs.http-client.experimental") (tk/defservice test-metric-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler (fn [_] {:status 200 :body "Hello, World!"}) "/hello") (add-ring-handler (fn [_] (do (Thread/sleep 5) {:status 200 :body "short"})) "/short") (add-ring-handler (fn [_] (do (Thread/sleep 100) {:status 200 :body "long"})) "/long") context)) (def hello-url "http://localhost:10000/hello") (def short-url "http://localhost:10000/short") (def long-url "http://localhost:10000/long") (def short-name (format "%s.with-url.%s.full-response" metric-namespace short-url)) (def short-name-with-get (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace short-url)) (def short-name-with-post (format "%s.with-url-and-method.%s.POST.full-response" metric-namespace short-url)) (def long-name (format "%s.with-url.%s.full-response" metric-namespace long-url)) (def long-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace long-url)) (def long-foo-name "puppetlabs.http-client.experimental.with-metric-id.foo.full-response") (def long-foo-bar-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.full-response") (def long-foo-bar-baz-name "puppetlabs.http-client.experimental.with-metric-id.foo.bar.baz.full-response") (def hello-name (format "%s.with-url.%s.full-response" metric-namespace hello-url)) (def hello-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace hello-url)) (deftest metrics-test-java-async (testing "metrics work with java async client" (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (let [metric-registry (MetricRegistry.) hello-request-opts (RequestOptions. hello-url) short-request-opts (RequestOptions. short-url) long-request-opts (doto (RequestOptions. long-url) (.setMetricId (into-array ["foo" "bar" "baz"])))] (with-open [client (Async/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry)))] (-> client (.get hello-request-opts) (.deref)) ; warm it up (let [short-response (-> client (.get short-request-opts) (.deref)) long-response (-> client (.get long-request-opts) (.deref))] (-> client (.post short-request-opts) (.deref)) (is (= 200 (.getStatus short-response))) (is (= "short" (slurp (.getBody short-response)))) (is (= 200 (.getStatus long-response))) (is (= "long" (slurp (.getBody long-response)))) (.timer metric-registry "fake") (let [client-metric-registry (.getMetricRegistry client) client-metrics (Metrics/getClientMetrics client-metric-registry) client-metrics-data (Metrics/getClientMetricsData client-metric-registry) url-metrics (.getUrlTimers client-metrics) url-and-method-metrics (.getUrlAndMethodTimers client-metrics) metric-id-metrics (.getMetricIdTimers client-metrics) url-metrics-data (.getUrlData client-metrics-data) url-and-method-metrics-data (.getUrlAndMethodData client-metrics-data) metric-id-metrics-data (.getMetricIdData client-metrics-data) all-metrics (.getMetrics metric-registry)] (testing ".getMetricRegistry returns the associated MetricRegistry" (is (instance? MetricRegistry client-metric-registry))) (testing "Metrics/getClientMetrics returns only http client metrics" (is (= 11 (count all-metrics))) (is (= 10 (+ (count url-metrics) (count url-and-method-metrics) (count metric-id-metrics)))) (is (= 10 (+ (count url-metrics-data) (count url-and-method-metrics-data) (count metric-id-metrics-data))))) (testing ".getClientMetrics returns a map of category to array of timers" (is (= (set (list hello-name short-name long-name)) (set (map #(.getMetricName %) url-metrics)) (set (map #(.getMetricName %) url-metrics-data)))) (is (= (set (list hello-name-with-method short-name-with-get short-name-with-post long-name-with-method)) (set (map #(.getMetricName %) url-and-method-metrics)) (set (map #(.getMetricName %) url-and-method-metrics-data)))) (is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name)) (set (map #(.getMetricName %) metric-id-metrics)) (set (map #(.getMetricName %) metric-id-metrics-data)))) (is (every? #(instance? ClientTimer %) url-metrics)) (is (every? #(instance? ClientTimer %) url-and-method-metrics)) (is (every? #(instance? ClientTimer %) metric-id-metrics))) (testing ".getClientMetricsData returns a map of metric category to arrays of metric data" (let [short-data (first (filter #(= short-name (.getMetricName %)) url-metrics-data)) short-data-get (first (filter #(= short-name-with-get (.getMetricName %)) url-and-method-metrics-data)) short-data-post (first (filter #(= short-name-with-post (.getMetricName %)) url-and-method-metrics-data)) long-data (first (filter #(= long-name (.getMetricName %)) url-metrics-data))] (is (every? #(instance? ClientMetricData %) (concat url-metrics-data url-and-method-metrics-data metric-id-metrics-data))) (is (= short-name (.getMetricName short-data))) (is (= 2 (.getCount short-data))) (is (<= 5 (.getMean short-data))) (is (<= 10 (.getAggregate short-data))) (is (= short-name-with-get (.getMetricName short-data-get))) (is (= 1 (.getCount short-data-get))) (is (<= 5 (.getMean short-data-get))) (is (<= 5 (.getAggregate short-data-get))) (is (= short-name-with-post (.getMetricName short-data-post))) (is (= 1 (.getCount short-data-post))) (is (<= 5 (.getMean short-data-post))) (is (<= 5 (.getAggregate short-data-post))) (is (>= 1 (Math/abs (- (.getAggregate short-data) (+ (.getAggregate short-data-get) (.getAggregate short-data-post)))))) (is (= long-name (.getMetricName long-data))) (is (= 1 (.getCount long-data))) (is (<= 100 (.getMean long-data))) (is (<= 100 (.getAggregate long-data))) (is (> (.getAggregate long-data) (.getAggregate short-data)))))))) (with-open [client (Async/createClient (ClientOptions.))] (testing ".getMetricRegistry returns nil if no metric registry passed in" (is (= nil (.getMetricRegistry client)))))))))) (deftest metrics-test-clojure-async (testing "metrics work with clojure async client" (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (let [metric-registry (MetricRegistry.)] (with-open [client (async/create-client {:metric-registry metric-registry})] @(common/get client hello-url) ; warm it up (let [short-response @(common/get client short-url {:as :text :metric-id ["foo" "bar" "baz"]}) long-response @(common/get client long-url)] @(common/post client short-url) (is (= {:status 200 :body "short"} (select-keys short-response [:status :body]))) (is (= 200 (:status long-response))) (is (= "long" (slurp (:body long-response)))) (.timer metric-registry "fake") (let [client-metric-registry (common/get-client-metric-registry client) client-metrics (metrics/get-client-metrics client-metric-registry) client-metrics-data (metrics/get-client-metrics-data client-metric-registry) url-metrics (:url client-metrics) url-and-method-metrics (:url-and-method client-metrics) metric-id-metrics (:metric-id client-metrics) url-metrics-data (:url client-metrics-data) url-and-method-metrics-data (:url-and-method client-metrics-data) metric-id-metrics-data (:metric-id client-metrics-data) all-metrics (.getMetrics metric-registry)] (testing "get-client-metric-registry returns the associated MetricRegistry" (is (instance? MetricRegistry client-metric-registry))) (testing "get-client-metrics and get-client-metrics data return only http client metrics" (is (= 11 (count all-metrics))) (is (= 10 (+ (count url-metrics) (count url-and-method-metrics) (count metric-id-metrics)))) (is (= 10 (+ (count url-metrics-data) (count url-and-method-metrics-data) (count metric-id-metrics-data))))) (testing "get-client-metrics returns a map of category to array of timers" (is (= (set (list hello-name short-name long-name)) (set (map #(.getMetricName %) url-metrics)) (set (map :metric-name url-metrics-data)))) (is (= (set (list hello-name-with-method short-name-with-get short-name-with-post long-name-with-method)) (set (map #(.getMetricName %) url-and-method-metrics)) (set (map :metric-name url-and-method-metrics-data)))) (is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name)) (set (map #(.getMetricName %) metric-id-metrics)) (set (map :metric-name metric-id-metrics-data)))) (is (every? #(instance? ClientTimer %) url-metrics)) (is (every? #(instance? ClientTimer %) url-and-method-metrics)) (is (every? #(instance? ClientTimer %) metric-id-metrics))) (testing "get-client-metrics-data returns a map of metric category to metric data" (let [short-data (first (filter #(= short-name (:metric-name %)) url-metrics-data)) short-data-get (first (filter #(= short-name-with-get (:metric-name %)) url-and-method-metrics-data)) short-data-post (first (filter #(= short-name-with-post (:metric-name %)) url-and-method-metrics-data)) long-data (first (filter #(= long-name (:metric-name %)) url-metrics-data))] (is (= short-name (:metric-name short-data))) (is (= 2 (:count short-data))) (is (<= 5 (:mean short-data))) (is (<= 10 (:aggregate short-data))) (is (= short-name-with-get (:metric-name short-data-get))) (is (= 1 (:count short-data-get))) (is (<= 5 (:mean short-data-get))) (is (<= 5 (:aggregate short-data-get))) (is (= short-name-with-post (:metric-name short-data-post))) (is (= 1 (:count short-data-post))) (is (<= 5 (:mean short-data-post))) (is (<= 5 (:aggregate short-data-post))) (is (>= 1 (Math/abs (- (:aggregate short-data) (+ (:aggregate short-data-get) (:aggregate short-data-post)))))) (is (= long-name (:metric-name long-data))) (is (= 1 (:count long-data))) (is (<= 100 (:mean long-data))) (is (<= 100 (:aggregate long-data))) (is (> (:mean long-data) (:mean short-data))))))))) (with-open [client (async/create-client {})] (testing "get-client-metric-registry returns nil if no metric registry passed in" (is (= nil (common/get-client-metric-registry client))))))))) (deftest metrics-test-java-sync (testing "metrics work with java sync client" (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (let [metric-registry (MetricRegistry.) hello-request-opts (RequestOptions. hello-url) short-request-opts (RequestOptions. short-url) long-request-opts (doto (RequestOptions. long-url) (.setMetricId (into-array ["foo" "bar" "baz"])))] (with-open [client (Sync/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry)))] (.get client hello-request-opts) ; warm it up (let [short-response (.get client short-request-opts) long-response (.get client long-request-opts)] (.post client short-request-opts) (is (= 200 (.getStatus short-response))) (is (= "short" (slurp (.getBody short-response)))) (is (= 200 (.getStatus long-response))) (is (= "long" (slurp (.getBody long-response)))) (.timer metric-registry "fake") (let [client-metric-registry (.getMetricRegistry client) client-metrics (Metrics/getClientMetrics client-metric-registry) client-metrics-data (Metrics/getClientMetricsData client-metric-registry) url-metrics (.getUrlTimers client-metrics) url-and-method-metrics (.getUrlAndMethodTimers client-metrics) metric-id-metrics (.getMetricIdTimers client-metrics) url-metrics-data (.getUrlData client-metrics-data) url-and-method-metrics-data (.getUrlAndMethodData client-metrics-data) metric-id-metrics-data (.getMetricIdData client-metrics-data) all-metrics (.getMetrics metric-registry)] (testing ".getMetricRegistry returns the associated MetricRegistry" (is (instance? MetricRegistry client-metric-registry))) (testing "Metrics/getClientMetrics returns only http client metrics" (is (= 11 (count all-metrics))) (is (= 10 (+ (count url-metrics) (count url-and-method-metrics) (count metric-id-metrics)))) (is (= 10 (+ (count url-metrics-data) (count url-and-method-metrics-data) (count metric-id-metrics-data))))) (testing ".getClientMetrics returns a map of category to array of timers" (is (= (set (list hello-name short-name long-name)) (set (map #(.getMetricName %) url-metrics)) (set (map #(.getMetricName %) url-metrics-data)))) (is (= (set (list hello-name-with-method short-name-with-get short-name-with-post long-name-with-method)) (set (map #(.getMetricName %) url-and-method-metrics)) (set (map #(.getMetricName %) url-and-method-metrics-data)))) (is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name)) (set (map #(.getMetricName %) metric-id-metrics)) (set (map #(.getMetricName %) metric-id-metrics-data)))) (is (every? #(instance? ClientTimer %) url-metrics)) (is (every? #(instance? ClientTimer %) url-and-method-metrics)) (is (every? #(instance? ClientTimer %) metric-id-metrics))) (testing ".getClientMetricsData returns a map of metric category to arrays of metric data" (let [short-data (first (filter #(= short-name (.getMetricName %)) url-metrics-data)) short-data-get (first (filter #(= short-name-with-get (.getMetricName %)) url-and-method-metrics-data)) short-data-post (first (filter #(= short-name-with-post (.getMetricName %)) url-and-method-metrics-data)) long-data (first (filter #(= long-name (.getMetricName %)) url-metrics-data))] (is (every? #(instance? ClientMetricData %) (concat url-metrics-data url-and-method-metrics-data metric-id-metrics-data))) (is (= short-name (.getMetricName short-data))) (is (= 2 (.getCount short-data))) (is (<= 5 (.getMean short-data))) (is (<= 10 (.getAggregate short-data))) (is (= short-name-with-get (.getMetricName short-data-get))) (is (= 1 (.getCount short-data-get))) (is (<= 5 (.getMean short-data-get))) (is (<= 5 (.getAggregate short-data-get))) (is (= short-name-with-post (.getMetricName short-data-post))) (is (= 1 (.getCount short-data-post))) (is (<= 5 (.getMean short-data-post))) (is (<= 5 (.getAggregate short-data-post))) (is (>= 1 (Math/abs (- (.getAggregate short-data) (+ (.getAggregate short-data-get) (.getAggregate short-data-post)))))) (is (= long-name (.getMetricName long-data))) (is (= 1 (.getCount long-data))) (is (<= 100 (.getMean long-data))) (is (<= 100 (.getAggregate long-data))) (is (> (.getMean long-data) (.getMean short-data)))))))) (with-open [client (Sync/createClient (ClientOptions.))] (testing ".getMetricRegistry returns nil if no metric registry passed in" (is (= nil (.getMetricRegistry client)))))))))) (deftest metrics-test-clojure-sync (testing "metrics work with clojure sync client" (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (let [metric-registry (MetricRegistry.)] (with-open [client (sync/create-client {:metric-registry metric-registry})] (common/get client hello-url) ; warm it up (let [short-response (common/get client short-url {:as :text}) long-response (common/get client long-url {:as :text :metric-id ["foo" "bar" "baz"]})] (common/post client short-url) (is (= {:status 200 :body "short"} (select-keys short-response [:status :body]))) (is (= {:status 200 :body "long"} (select-keys long-response [:status :body]))) (.timer metric-registry "fake") (let [client-metric-registry (common/get-client-metric-registry client) client-metrics (metrics/get-client-metrics client-metric-registry) client-metrics-data (metrics/get-client-metrics-data client-metric-registry) url-metrics (:url client-metrics) url-and-method-metrics (:url-and-method client-metrics) metric-id-metrics (:metric-id client-metrics) url-metrics-data (:url client-metrics-data) url-and-method-metrics-data (:url-and-method client-metrics-data) metric-id-metrics-data (:metric-id client-metrics-data) all-metrics (.getMetrics metric-registry)] (testing "get-client-metric-registry returns the associated MetricRegistry" (is (instance? MetricRegistry client-metric-registry))) (testing "get-client-metrics and get-client-metrics data return only http client metrics" (is (= 11 (count all-metrics))) (is (= 10 (+ (count url-metrics) (count url-and-method-metrics) (count metric-id-metrics)))) (is (= 10 (+ (count url-metrics-data) (count url-and-method-metrics-data) (count metric-id-metrics-data))))) (testing "get-client-metrics returns a map of category to array of timers" (is (= (set (list hello-name short-name long-name)) (set (map #(.getMetricName %) url-metrics)) (set (map :metric-name url-metrics-data)))) (is (= (set (list hello-name-with-method short-name-with-get short-name-with-post long-name-with-method)) (set (map #(.getMetricName %) url-and-method-metrics)) (set (map :metric-name url-and-method-metrics-data)))) (is (= (set (list long-foo-name long-foo-bar-name long-foo-bar-baz-name)) (set (map #(.getMetricName %) metric-id-metrics)) (set (map :metric-name metric-id-metrics-data)))) (is (every? #(instance? ClientTimer %) url-metrics)) (is (every? #(instance? ClientTimer %) url-and-method-metrics)) (is (every? #(instance? ClientTimer %) metric-id-metrics))) (testing "get-client-metrics-data returns a map of metric category to metric data" (let [short-data (first (filter #(= short-name (:metric-name %)) url-metrics-data)) short-data-get (first (filter #(= short-name-with-get (:metric-name %)) url-and-method-metrics-data)) short-data-post (first (filter #(= short-name-with-post (:metric-name %)) url-and-method-metrics-data)) long-data (first (filter #(= long-name (:metric-name %)) url-metrics-data))] (is (= short-name (:metric-name short-data))) (is (= 2 (:count short-data))) (is (<= 5 (:mean short-data))) (is (<= 10 (:aggregate short-data))) (is (= short-name-with-get (:metric-name short-data-get))) (is (= 1 (:count short-data-get))) (is (<= 5 (:mean short-data-get))) (is (<= 5 (:aggregate short-data-get))) (is (= short-name-with-post (:metric-name short-data-post))) (is (= 1 (:count short-data-post))) (is (<= 5 (:mean short-data-post))) (is (<= 5 (:aggregate short-data-post))) (is (>= 1 (Math/abs (- (:aggregate short-data) (+ (:aggregate short-data-get) (:aggregate short-data-post)))))) (is (= long-name (:metric-name long-data))) (is (= 1 (:count long-data))) (is (<= 100 (:mean long-data))) (is (<= 100 (:aggregate long-data))) (is (> (:mean long-data) (:mean short-data)))))))) (with-open [client (sync/create-client {})] (testing "get-client-metric-registry returns nil if no metric registry passed in" (is (= nil (common/get-client-metric-registry client)))))))))) (deftest java-metrics-for-unbuffered-streaming-test (testlogging/with-test-logging (let [data (unbuffered-test/generate-data (* 1024 1024))] (testing "metrics work for a successful request" (let [metric-registry (MetricRegistry.)] (testwebserver/with-test-webserver-and-config (unbuffered-test/successful-handler data nil) port {:shutdown-timeout-seconds 1} (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 20000) (.setConnectTimeoutMilliseconds 100) (.setMetricRegistry metric-registry) (Async/createClient))] (let [url (str "http://localhost:" port "/hello") request-options (doto (RequestOptions. url) (.setAs ResponseBodyType/UNBUFFERED_STREAM)) response (-> client (.get request-options) .deref) status (.getStatus response) body (.getBody response)] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4)] (.read instream buf) (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (Thread/sleep 1000) ;; check that the full-response metric takes this into account (is (= (str data "yyyy") (str "xxxx" (slurp instream))))) ;; Read the rest and validate (let [client-metric-registry (.getMetricRegistry client) client-metrics (Metrics/getClientMetrics client-metric-registry) client-metrics-data (Metrics/getClientMetricsData client-metric-registry) full-response-name (format "%s.with-url.%s.full-response" metric-namespace url) full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace url)] (is (= [full-response-name] (map #(.getMetricName %) (.getUrlTimers client-metrics)) (map #(.getMetricName %) (.getUrlData client-metrics-data)))) (is (= [full-response-name-with-method] (map #(.getMetricName %) (.getUrlAndMethodTimers client-metrics)) (map #(.getMetricName %) (.getUrlAndMethodData client-metrics-data)))) (is (= [] (.getMetricIdTimers client-metrics) (.getMetricIdData client-metrics-data))) (is (every? #(instance? ClientTimer %) (concat (.getUrlTimers client-metrics) (.getUrlAndMethodTimers client-metrics)))) (let [full-response-data (first (.getUrlData client-metrics-data))] (is (every? #(instance? ClientMetricData %) (concat (.getUrlData client-metrics-data) (.getUrlAndMethodData client-metrics-data)))) (is (= 1 (.getCount full-response-data))) (is (= full-response-name (.getMetricName full-response-data))) (is (<= 1000 (.getMean full-response-data))) (is (<= 1000 (.getAggregate full-response-data)))))))))) (testing "metrics work for failed request" (try (testwebserver/with-test-webserver-and-config (unbuffered-test/blocking-handler data) port {:shutdown-timeout-seconds 1} (let [metric-registry (MetricRegistry.)] (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 200) (.setConnectTimeoutMilliseconds 100) (.setMetricRegistry metric-registry) (Async/createClient))] (let [url (str "http://localhost:" port "/hello") request-options (doto (RequestOptions. url) (.setAs ResponseBodyType/UNBUFFERED_STREAM)) response (-> client (.get request-options) .deref) error (.getError response) body (.getBody response)] (is (nil? error)) (is (thrown? SocketTimeoutException (slurp body))) (let [client-metric-registry (.getMetricRegistry client) client-metrics (Metrics/getClientMetrics client-metric-registry) client-metrics-data (Metrics/getClientMetricsData client-metric-registry) full-response-name (format "%s.with-url.%s.full-response" metric-namespace url) full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace url)] (is (= [full-response-name] (map #(.getMetricName %) (.getUrlTimers client-metrics)) (map #(.getMetricName %) (.getUrlData client-metrics-data)))) (is (= [full-response-name-with-method] (map #(.getMetricName %) (.getUrlAndMethodTimers client-metrics)) (map #(.getMetricName %) (.getUrlAndMethodData client-metrics-data)))) (is (= [] (.getMetricIdTimers client-metrics) (.getMetricIdData client-metrics-data))) (is (every? #(instance? ClientTimer %) (concat (.getUrlTimers client-metrics) (.getUrlAndMethodTimers client-metrics)))) (let [full-response-data (first (.getUrlData client-metrics-data))] (is (every? #(instance? ClientMetricData %) (concat (.getUrlData client-metrics-data) (.getUrlAndMethodData client-metrics-data)))) (is (= 1 (.getCount full-response-data))) (is (= full-response-name (.getMetricName full-response-data))) (is (<= 200 (.getMean full-response-data))) (is (<= 200 (.getAggregate full-response-data))))))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated )))))) (deftest clojure-metrics-for-unbuffered-streaming-test (testlogging/with-test-logging (let [data (unbuffered-test/generate-data (* 1024 1024)) opts {:as :unbuffered-stream}] (testing "metrics work for a successful request" (let [metric-registry (MetricRegistry.)] (testwebserver/with-test-webserver-and-config (unbuffered-test/successful-handler data nil) port {:shutdown-timeout-seconds 1} (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 20000 :metric-registry metric-registry})] (let [url (str "http://localhost:" port "/hello") response @(common/get client url opts) {:keys [status body]} response] (is (= 200 status)) (let [instream body buf (make-array Byte/TYPE 4)] (.read instream buf) (is (= "xxxx" (String. buf "UTF-8"))) ;; Make sure we can read a few chars off of the stream (Thread/sleep 1000) ;; check that the full-response metric takes this into account (is (= (str data "yyyy") (str "xxxx" (slurp instream))))) ;; Read the rest and validate (let [client-metric-registry (common/get-client-metric-registry client) client-metrics (metrics/get-client-metrics client-metric-registry) client-metrics-data (metrics/get-client-metrics-data client-metric-registry) full-response-name (format "%s.with-url.%s.full-response" metric-namespace url) full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace url)] (is (= [full-response-name] (map #(.getMetricName %) (:url client-metrics)) (map :metric-name (:url client-metrics-data)))) (is (= [full-response-name-with-method] (map #(.getMetricName %) (:url-and-method client-metrics)) (map :metric-name (:url-and-method client-metrics-data)))) (is (= [] (:metric-id client-metrics) (:metric-id client-metrics-data))) (is (every? #(instance? ClientTimer %) (concat (:url client-metrics) (:url-and-method client-metrics)))) (let [full-response-data (first (:url client-metrics-data))] (is (= {:count 1 :metric-name full-response-name} (select-keys full-response-data [:metric-name :count]))) (is (<= 1000 (:mean full-response-data))) (is (<= 1000 (:aggregate full-response-data)))))))))) (testing "metrics work for a failed request" (try (testwebserver/with-test-webserver-and-config (unbuffered-test/blocking-handler data) port {:shutdown-timeout-seconds 1} (let [metric-registry (MetricRegistry.) url (str "http://localhost:" port "/hello")] (with-open [client (async/create-client {:connect-timeout-milliseconds 100 :socket-timeout-milliseconds 200 :metric-registry metric-registry})] (let [response @(common/get client url opts) {:keys [body error]} response] (is (nil? error)) ;; Consume the body to get the exception (is (thrown? SocketTimeoutException (slurp body)))) (let [client-metric-registry (common/get-client-metric-registry client) client-metrics (metrics/get-client-metrics client-metric-registry) client-metrics-data (metrics/get-client-metrics-data client-metric-registry) full-response-name (format "%s.with-url.%s.full-response" metric-namespace url) full-response-name-with-method (format "%s.with-url-and-method.%s.GET.full-response" metric-namespace url)] (is (= [full-response-name] (map #(.getMetricName %) (:url client-metrics)) (map :metric-name (:url client-metrics-data)))) (is (= [full-response-name-with-method] (map #(.getMetricName %) (:url-and-method client-metrics)) (map :metric-name (:url-and-method client-metrics-data)))) (is (= [] (:metric-id client-metrics) (:metric-id client-metrics-data))) (is (every? #(instance? ClientTimer %) (concat (:url client-metrics) (:url-and-method client-metrics)))) (let [full-response-data (first (:url client-metrics-data))] (is (= {:count 1 :metric-name full-response-name} (select-keys full-response-data [:metric-name :count]))) (is (<= 200 (:mean full-response-data))) (is (<= 200 (:aggregate full-response-data)))))))) (catch TimeoutException e ;; Expected whenever a server-side failure is generated )))))) (deftest metric-namespace-test (let [metric-prefix "my-metric-prefix" server-id "my-server" metric-name-with-prefix (format "%s.http-client.experimental.with-url.%s.full-response" metric-prefix hello-url) metric-name-with-server-id (format "puppetlabs.%s.http-client.experimental.with-url.%s.full-response" server-id hello-url) get-metric-name (fn [metric-registry] (.getMetricName (first (Metrics/getClientMetricsDataByUrl metric-registry hello-url))))] (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (testing "custom metric namespace works for java async client" (testing "metric prefix works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-metric-prefix (Async/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setMetricPrefix metric-prefix)))] (is (= (format "%s.http-client.experimental" metric-prefix) (.getMetricNamespace client-with-metric-prefix))) (-> client-with-metric-prefix (.get (RequestOptions. hello-url))) (is (= metric-name-with-prefix (get-metric-name metric-registry)))))) (testing "server id works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (Async/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setServerId server-id)))] (-> client-with-server-id (.get (RequestOptions. hello-url))) (is (= metric-name-with-server-id (get-metric-name metric-registry)))))) (testing "metric prefix overrides server id if both are set" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (Async/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setMetricPrefix metric-prefix) (.setServerId server-id)))] (-> client-with-server-id (.get (RequestOptions. hello-url))) (is (= metric-name-with-prefix (get-metric-name metric-registry))) (is (logged? #"Metric prefix and server id both set.*" :warn)))))) (testing "custom metric namespace works for clojure async client" (testing "metric prefix works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-metric-prefix (async/create-client {:metric-registry metric-registry :metric-prefix metric-prefix})] (is (= (format "%s.http-client.experimental" metric-prefix) (common/get-client-metric-namespace client-with-metric-prefix))) (-> client-with-metric-prefix (common/get hello-url)) (is (= metric-name-with-prefix (get-metric-name metric-registry)))))) (testing "server id works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (async/create-client {:metric-registry metric-registry :server-id server-id})] (-> client-with-server-id (common/get hello-url)) (is (= metric-name-with-server-id (get-metric-name metric-registry)))))) (testing "metric prefix overrides server id if both are set" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (async/create-client {:metric-registry metric-registry :metric-prefix metric-prefix :server-id server-id})] (-> client-with-server-id (common/get hello-url)) (is (= metric-name-with-prefix (get-metric-name metric-registry))))))) (testing "custom metric namespace works for Java sync client" (testing "metric prefix works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-metric-prefix (Sync/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setMetricPrefix metric-prefix)))] (is (= (format "%s.http-client.experimental" metric-prefix) (.getMetricNamespace client-with-metric-prefix))) (-> client-with-metric-prefix (.get (RequestOptions. hello-url))) (is (= metric-name-with-prefix (get-metric-name metric-registry)))))) (testing "server id works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (Sync/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setServerId server-id)))] (-> client-with-server-id (.get (RequestOptions. hello-url))) (is (= metric-name-with-server-id (get-metric-name metric-registry)))))) (testing "metric prefix overrides server id if both are set" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (Sync/createClient (doto (ClientOptions.) (.setMetricRegistry metric-registry) (.setMetricPrefix metric-prefix) (.setServerId server-id)))] (-> client-with-server-id (.get (RequestOptions. hello-url))) (is (= metric-name-with-prefix (get-metric-name metric-registry))))))) (testing "custom metric namespace works for clojure sync client" (testing "metric prefix works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-metric-prefix (sync/create-client {:metric-registry metric-registry :metric-prefix metric-prefix})] (is (= (format "%s.http-client.experimental" metric-prefix) (common/get-client-metric-namespace client-with-metric-prefix))) (-> client-with-metric-prefix (common/get hello-url)) (is (= metric-name-with-prefix (get-metric-name metric-registry)))))) (testing "server id works" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (sync/create-client {:metric-registry metric-registry :server-id server-id})] (-> client-with-server-id (common/get hello-url)) (is (= metric-name-with-server-id (get-metric-name metric-registry)))))) (testing "metric prefix overrides server id if both are set" (let [metric-registry (MetricRegistry.)] (with-open [client-with-server-id (sync/create-client {:metric-registry metric-registry :metric-prefix metric-prefix :server-id server-id})] (-> client-with-server-id (common/get hello-url)) (is (= metric-name-with-prefix (get-metric-name metric-registry))))))))))) (deftest toggleable-url-metrics (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-metric-web-service] {:webserver {:port 10000}} (testing "url-metrics can be disabled for clojure async client" (with-open [client (async/create-client {:metric-registry (MetricRegistry.) :enable-url-metrics? false})] (let [response @(common/get client hello-url {:metric-id ["foo" "bar"]})] (is (= 200 (:status response))) (let [client-metrics (-> client (common/get-client-metric-registry) (metrics/get-client-metrics)) metric-names (set (map #(.getMetricName %) (:metric-id client-metrics)))] (is (empty? (:url client-metrics))) (is (empty? (:url-and-method client-metrics))) (is (= #{long-foo-name long-foo-bar-name} metric-names)))))) (testing "url-metrics can be disabled for java async client" (with-open [client (Async/createClient (doto (ClientOptions.) (.setMetricRegistry (MetricRegistry.)) (.setEnableURLMetrics false)))] (let [request-opts (doto (RequestOptions. hello-url) (.setMetricId (into-array ["foo" "bar"]))) response (-> client (.get request-opts) (.deref))] (is (= 200 (.getStatus response))) (let [client-metrics (-> client (.getMetricRegistry) (Metrics/getClientMetrics)) metric-names (set (map #(.getMetricName %) (.getMetricIdTimers client-metrics)))] (is (.isEmpty (.getUrlTimers client-metrics))) (is (.isEmpty (.getUrlAndMethodTimers client-metrics))) (is (= #{long-foo-name long-foo-bar-name} metric-names)))))) (testing "url-metrics can be disabled for clojure sync client" (with-open [client (sync/create-client {:metric-registry (MetricRegistry.) :enable-url-metrics? false})] (let [response (common/get client hello-url {:metric-id ["foo" "bar"]})] (is (= 200 (:status response))) (let [client-metrics (-> client (common/get-client-metric-registry) (metrics/get-client-metrics)) metric-names (set (map #(.getMetricName %) (:metric-id client-metrics)))] (is (empty? (:url client-metrics))) (is (empty? (:url-and-method client-metrics))) (is (= #{long-foo-name long-foo-bar-name} metric-names)))))) (testing "url-metrics can be disabled for java sync client" (with-open [client (Sync/createClient (doto (ClientOptions.) (.setMetricRegistry (MetricRegistry.)) (.setEnableURLMetrics false)))] (let [request-opts (doto (RequestOptions. hello-url) (.setMetricId (into-array ["foo" "bar"]))) response (-> client (.get request-opts))] (is (= 200 (.getStatus response))) (let [client-metrics (-> client (.getMetricRegistry) (Metrics/getClientMetrics)) metric-names (set (map #(.getMetricName %) (.getMetricIdTimers client-metrics)))] (is (.isEmpty (.getUrlTimers client-metrics))) (is (.isEmpty (.getUrlAndMethodTimers client-metrics))) (is (= #{long-foo-name long-foo-bar-name} metric-names)))))))))clj-http-client-0.9.0/test/puppetlabs/http/client/sync_plaintext_test.clj000066400000000000000000001145741312026620400267210ustar00rootroot00000000000000(ns puppetlabs.http.client.sync-plaintext-test (:import (com.puppetlabs.http.client Sync RequestOptions SimpleRequestOptions ResponseBodyType ClientOptions HttpClientException) (java.io ByteArrayInputStream InputStream) (org.apache.http.impl.nio.client HttpAsyncClients) (java.net ConnectException ServerSocket SocketTimeoutException URI)) (:require [clojure.test :refer :all] [puppetlabs.http.client.test-common :refer :all] [puppetlabs.trapperkeeper.core :as tk] [puppetlabs.trapperkeeper.testutils.bootstrap :as testutils] [puppetlabs.trapperkeeper.testutils.logging :as testlogging] [puppetlabs.trapperkeeper.testutils.webserver :as testwebserver] [puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9] [puppetlabs.http.client.sync :as sync] [puppetlabs.http.client.common :as common] [schema.test :as schema-test] [clojure.java.io :as io] [ring.middleware.cookies :refer [wrap-cookies]])) (use-fixtures :once schema-test/validate-schemas) (defn app [_] {:status 200 :body "Hello, World!"}) (defn cookie-handler [_] {:status 200 :body "cookie has been set" :cookies {"session_id" {:value "session-id-hash"}}}) (defn check-cookie-handler [req] (if (empty? (get req :cookies)) {:status 400 :body "cookie has not been set"} {:status 200 :body "cookie has been set"})) (tk/defservice test-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler app "/hello") context)) (tk/defservice test-cookie-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler (wrap-cookies cookie-handler) "/cookietest") (add-ring-handler (wrap-cookies check-cookie-handler) "/cookiecheck") context)) (defn basic-test [http-method java-method clj-fn] (testing (format "sync client: HTTP method: '%s'" http-method) (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (testing "java sync client" (let [request-options (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) response (java-method request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (clj-fn "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))))))) (deftest sync-client-head-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (testing "java sync client" (let [request-options (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) response (Sync/head request-options)] (is (= 200 (.getStatus response))) (is (= nil (.getBody response))))) (testing "clojure sync client" (let [response (sync/head "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= nil (:body response)))))))) (deftest sync-client-get-test (basic-test "GET" #(Sync/get %) sync/get)) (deftest sync-client-post-test (basic-test "POST" #(Sync/post %) sync/post)) (deftest sync-client-put-test (basic-test "PUT" #(Sync/put %) sync/put)) (deftest sync-client-delete-test (basic-test "DELETE" #(Sync/delete %) sync/delete)) (deftest sync-client-trace-test (basic-test "TRACE" #(Sync/trace %) sync/trace)) (deftest sync-client-options-test (basic-test "OPTIONS" #(Sync/options %) sync/options)) (deftest sync-client-patch-test (basic-test "PATCH" #(Sync/patch %) sync/patch)) (deftest sync-client-persistent-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (testing "persistent java client" (let [request-options (RequestOptions. "http://localhost:10000/hello/") client-options (ClientOptions.) client (Sync/createClient client-options)] (testing "HEAD request with persistent sync client" (let [response (.head client request-options)] (is (= 200 (.getStatus response))) (is (= nil (.getBody response))))) (testing "GET request with persistent sync client" (let [response (.get client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "POST request with persistent sync client" (let [response (.post client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "PUT request with persistent sync client" (let [response (.put client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "DELETE request with persistent sync client" (let [response (.delete client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "TRACE request with persistent sync client" (let [response (.trace client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "OPTIONS request with persistent sync client" (let [response (.options client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "PATCH request with persistent sync client" (let [response (.patch client request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "client closes properly" (.close client) (is (thrown? IllegalStateException (.get client request-options)))))) (testing "persistent clojure client" (let [client (sync/create-client {})] (testing "HEAD request with persistent sync client" (let [response (common/head client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= nil (:body response))))) (testing "GET request with persistent sync client" (let [response (common/get client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "POST request with persistent sync client" (let [response (common/post client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "PUT request with persistent sync client" (let [response (common/put client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "DELETE request with persistent sync client" (let [response (common/delete client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "TRACE request with persistent sync client" (let [response (common/trace client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "OPTIONS request with persistent sync client" (let [response (common/options client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "PATCH request with persistent sync client" (let [response (common/patch client "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "GET request via request function with persistent sync client" (let [response (common/make-request client "http://localhost:10000/hello/" :get)] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "Bad verb request via request function with persistent sync client" (is (thrown? IllegalArgumentException (common/make-request client "http://localhost:10000/hello/" :bad)))) (testing "client closes properly" (common/close client) (is (thrown? IllegalStateException (common/get client "http://localhost:10000/hello"))))))))) (deftest sync-client-as-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (testing "java sync client: :as unspecified" (let [request-options (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (instance? InputStream (.getBody response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "java sync client: :as :stream" (let [request-options (.. (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (setAs ResponseBodyType/STREAM)) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (instance? InputStream (.getBody response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "java sync client: :as :text" (let [request-options (.. (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (setAs ResponseBodyType/TEXT)) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (string? (.getBody response))) (is (= "Hello, World!" (.getBody response))))) (testing "clojure sync client: :as unspecified" (let [response (sync/get "http://localhost:10000/hello/")] (is (= 200 (:status response))) (is (instance? InputStream (:body response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "clojure sync client: :as :stream" (let [response (sync/get "http://localhost:10000/hello/" {:as :stream})] (is (= 200 (:status response))) (is (instance? InputStream (:body response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "clojure sync client: :as :text" (let [response (sync/get "http://localhost:10000/hello/" {:as :text})] (is (= 200 (:status response))) (is (string? (:body response))) (is (= "Hello, World!" (:body response)))))))) (deftest request-with-client-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:port 10000}} (let [client (HttpAsyncClients/createDefault) opts {:method :get :url "http://localhost:10000/hello/"}] (.start client) (testing "GET request works with request-with-client" (let [response (sync/request-with-client opts client)] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (testing "Client persists when passed to request-with-client" (let [response (sync/request-with-client opts client)] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))) (.close client))))) (deftest java-api-cookie-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-cookie-service] {:webserver {:port 10000}} (let [client (Sync/createClient (ClientOptions.))] (testing "Set a cookie using Java API" (let [response (.get client (RequestOptions. "http://localhost:10000/cookietest"))] (is (= 200 (.getStatus response))))) (testing "Check if cookie still exists" (let [response (.get client (RequestOptions. "http://localhost:10000/cookiecheck"))] (is (= 200 (.getStatus response))))))))) (deftest clj-api-cookie-test (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-cookie-service] {:webserver {:port 10000}} (let [client (sync/create-client {})] (testing "Set a cookie using Clojure API" (let [response (common/get client "http://localhost:10000/cookietest")] (is (= 200 (:status response))))) (testing "Check if cookie still exists" (let [response (common/get client "http://localhost:10000/cookiecheck")] (is (= 200 (:status response))))))))) (defn header-app [req] (let [val (get-in req [:headers "fooheader"])] {:status 200 :headers {"myrespheader" val} :body val})) (tk/defservice test-header-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler header-app "/hello") context)) (deftest sync-client-request-headers-test (testlogging/with-test-logging (testutils/with-app-with-config header-app [jetty9/jetty9-service test-header-web-service] {:webserver {:port 10000}} (testing "java sync client" (let [request-options (-> (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (.setHeaders {"fooheader" "foo"})) response (Sync/post request-options)] (is (= 200 (.getStatus response))) (is (= "foo" (slurp (.getBody response)))) (is (= "foo" (-> (.getHeaders response) (.get "myrespheader")))))) (testing "clojure sync client" (let [response (sync/post "http://localhost:10000/hello/" {:headers {"fooheader" "foo"}})] (is (= 200 (:status response))) (is (= "foo" (slurp (:body response)))) (is (= "foo" (get-in response [:headers "myrespheader"])))))))) (defn req-body-app [req] {:status 200 :headers (if-let [content-type (:content-type req)] {"Content-Type" (:content-type req)}) :body (slurp (:body req))}) (tk/defservice test-body-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler req-body-app "/hello") context)) (defn- validate-java-request [body-to-send headers-to-send expected-content-type expected-response-body] (let [request-options (-> (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (.setBody body-to-send) (.setHeaders headers-to-send)) response (Sync/post request-options)] (is (= 200 (.getStatus response))) (is (= (-> (.getHeaders response) (.get "content-type")) expected-content-type)) (is (= expected-response-body (slurp (.getBody response)))))) (defn- validate-clj-request [body-to-send headers-to-send expected-content-type expected-response-body] (let [response (sync/post "http://localhost:10000/hello/" {:body body-to-send :headers headers-to-send})] (is (= 200 (:status response))) (is (= (get-in response [:headers "content-type"]) expected-content-type)) (is (= expected-response-body (slurp (:body response)))))) (deftest sync-client-request-body-test (testlogging/with-test-logging (testutils/with-app-with-config req-body-app [jetty9/jetty9-service test-body-web-service] {:webserver {:port 10000}} (testing "java sync client: string body for post request with explicit content type and UTF-8 encoding uses UTF-8 encoding" (validate-java-request "foo�" {"Content-Type" "text/plain; charset=utf-8"} "text/plain; charset=UTF-8" "foo�")) (testing "java sync client: string body for post request with explicit content type and ISO-8859-1 encoding uses ISO-8859-1 encoding" (validate-java-request "foo�" {"Content-Type" "text/plain; charset=iso-8859-1"} "text/plain; charset=ISO-8859-1" "foo?")) (testing "java sync client: string body for post request with explicit content type but without explicit encoding uses UTF-8 encoding" (validate-java-request "foo�" {"Content-Type" "text/plain"} "text/plain; charset=UTF-8" "foo�")) (testing "java sync client: string body for post request without explicit content or encoding uses ISO-8859-1 encoding" (validate-java-request "foo�" nil "text/plain; charset=ISO-8859-1" "foo?")) (testing "java sync client: input stream body for post request" (let [request-options (-> (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (.setBody (ByteArrayInputStream. (.getBytes "foo�" "UTF-8"))) (.setHeaders {"Content-Type" "text/plain; charset=UTF-8"})) response (Sync/post request-options)] (is (= 200 (.getStatus response))) (is (= "foo�" (slurp (.getBody response)))))) (testing "clojure sync client: string body for post request with explicit content type and UTF-8 encoding uses UTF-8 encoding" (validate-clj-request "foo�" {"content-type" "text/plain; charset=utf-8"} "text/plain; charset=UTF-8" "foo�")) (testing "clojure sync client: string body for post request with explicit content type and ISO-8859 encoding uses ISO-8859-1 encoding" (validate-clj-request "foo�" {"content-type" "text/plain; charset=iso-8859-1"} "text/plain; charset=ISO-8859-1" "foo?")) (testing "clojure sync client: string body for post request with explicit content type but without explicit encoding uses UTF-8 encoding" (validate-clj-request "foo�" {"content-type" "text/plain"} "text/plain; charset=UTF-8" "foo�")) (testing "clojure sync client: string body for post request without explicit content type or encoding uses ISO-8859-1 encoding" (validate-clj-request "foo�" {} "text/plain; charset=ISO-8859-1" "foo?")) (testing "clojure sync client: input stream body for post request" (let [response (sync/post "http://localhost:10000/hello/" {:body (io/input-stream (.getBytes "foo�" "UTF-8")) :headers {"content-type" "text/plain; charset=UTF-8"}})] (is (= 200 (:status response))) (is (= "foo�" (slurp (:body response))))))))) (def compressible-body (apply str (repeat 1000 "f"))) (defn compression-app [req] {:status 200 :headers {"orig-accept-encoding" (get-in req [:headers "accept-encoding"]) "content-type" "text/plain" "charset" "UTF-8"} :body compressible-body}) (tk/defservice test-compression-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler compression-app "/hello") context)) (defn test-compression [desc opts accept-encoding content-encoding content-should-match?] (testlogging/with-test-logging (testutils/with-app-with-config req-body-app [jetty9/jetty9-service test-compression-web-service] {:webserver {:port 10000}} (testing (str "java sync client: compression headers / response: " desc) (let [request-opts (cond-> (SimpleRequestOptions. (URI. "http://localhost:10000/hello/")) (contains? opts :decompress-body) (.setDecompressBody (:decompress-body opts)) (contains? opts :headers) (.setHeaders (:headers opts))) response (Sync/get request-opts)] (is (= 200 (.getStatus response))) (is (= accept-encoding (.. response getHeaders (get "orig-accept-encoding")))) (is (= content-encoding (.. response getOrigContentEncoding))) (if content-should-match? (is (= compressible-body (slurp (.getBody response)))) (is (not= compressible-body (slurp (.getBody response))))))) (testing (str "clojure sync client: compression headers / response: " desc) (let [response (sync/post "http://localhost:10000/hello/" opts)] (is (= 200 (:status response))) (is (= accept-encoding (get-in response [:headers "orig-accept-encoding"]))) (is (= content-encoding (:orig-content-encoding response))) (if content-should-match? (is (= compressible-body (slurp (:body response)))) (is (not= compressible-body (slurp (:body response)))))))))) (deftest sync-client-compression-test (test-compression "default" {} "gzip, deflate" "gzip" true)) (deftest sync-client-compression-gzip-test (test-compression "explicit gzip" {:headers {"accept-encoding" "gzip"}} "gzip" "gzip" true)) (deftest sync-client-compression-disabled-test (test-compression "explicit disable" {:decompress-body false} nil nil true)) (deftest sync-client-decompression-disabled-test (test-compression "explicit disable" {:headers {"accept-encoding" "gzip"} :decompress-body false} "gzip" "gzip" false)) (deftest query-params-test-sync (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-params-web-service] {:webserver {:port 8080}} (testing "URL Query Parameters work with the Java client" (let [request-options (SimpleRequestOptions. (URI. "http://localhost:8080/params?foo=bar&baz=lux"))] (let [response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (= queryparams (read-string (slurp (.getBody response)))))))) (testing "URL Query Parameters work with the clojure client" (let [opts {:method :get :url "http://localhost:8080/params/" :query-params queryparams :as :text} response (sync/get "http://localhost:8080/params" opts)] (is (= 200 (:status response))) (is (= queryparams (read-string (:body response))))))))) (deftest redirect-test-sync (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service redirect-web-service] {:webserver {:port 8080}} (testing (str "redirects on POST not followed by Java client " "when forceRedirects option not set to true") (let [request-options (SimpleRequestOptions. (URI. "http://localhost:8080/hello")) response (Sync/post request-options)] (is (= 302 (.getStatus response))))) (testing "redirects on POST followed by Java client when option is set" (let [request-options (.. (SimpleRequestOptions. (URI. "http://localhost:8080/hello")) (setForceRedirects true)) response (Sync/post request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "redirects not followed by Java client when :follow-redirects is false" (let [request-options (.. (SimpleRequestOptions. (URI. "http://localhost:8080/hello")) (setFollowRedirects false)) response (Sync/get request-options)] (is (= 302 (.getStatus response))))) (testing ":follow-redirects overrides :force-redirects for Java client" (let [request-options (.. (SimpleRequestOptions. (URI. "http://localhost:8080/hello")) (setFollowRedirects false) (setForceRedirects true)) response (Sync/get request-options)] (is (= 302 (.getStatus response))))) (testing (str "redirects on POST not followed by clojure client " "when :force-redirects is not set to true") (let [opts {:method :post :url "http://localhost:8080/hello" :as :text :force-redirects false} response (sync/post "http://localhost:8080/hello" opts)] (is (= 302 (:status response))))) (testing "redirects on POST followed by clojure client when option is set" (let [opts {:method :post :url "http://localhost:8080/hello" :as :text :force-redirects true} response (sync/post "http://localhost:8080/hello" opts)] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))) (testing (str "redirects not followed by clojure client when :follow-redirects " "is set to false") (let [response (sync/get "http://localhost:8080/hello" {:as :text :follow-redirects false})] (is (= 302 (:status response))))) (testing ":follow-redirects overrides :force-redirects with clojure client" (let [response (sync/get "http://localhost:8080/hello" {:as :text :follow-redirects false :force-redirects true})] (is (= 302 (:status response))))) (testing (str "redirects on POST followed by persistent clojure client " "when option is set") (let [client (sync/create-client {:force-redirects true}) response (common/post client "http://localhost:8080/hello" {:as :text})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))) (common/close client))) (testing (str "persistent clojure client does not follow redirects when " ":follow-redirects is set to false") (let [client (sync/create-client {:follow-redirects false}) response (common/get client "http://localhost:8080/hello" {:as :text})] (is (= 302 (:status response))) (common/close client))) (testing ":follow-redirects overrides :force-redirects with persistent clj client" (let [client (sync/create-client {:follow-redirects false :force-redirects true}) response (common/get client "http://localhost:8080/hello" {:as :text})] (is (= 302 (:status response))) (common/close client)))))) (defmacro wrapped-connect-exception-thrown? [& body] `(try (testlogging/with-test-logging ~@body) (throw (IllegalStateException. "Expected HttpClientException but none thrown!")) (catch HttpClientException e# (if-let [cause# (.getCause e#)] (or (instance? SocketTimeoutException cause#) (instance? ConnectException cause#) (throw (IllegalStateException. (str "Expected SocketTimeoutException or ConnectException " "cause but found: " cause#)))) (throw (IllegalStateException. (str "Expected SocketTimeoutException or ConnectException but " "no cause found. Message:" (.getMessage e#)))))))) (defmacro wrapped-timeout-exception-thrown? [& body] `(try (testlogging/with-test-logging ~@body) (throw (IllegalStateException. "Expected HttpClientException but none thrown!")) (catch HttpClientException e# (if-let [cause# (.getCause e#)] (or (instance? SocketTimeoutException cause#) (throw (IllegalStateException. (str "Expected SocketTimeoutException cause but found: " cause#)))) (throw (IllegalStateException. (str "Expected SocketTimeoutException but no cause found. " "Message: " (.getMessage e#)))))))) (deftest short-connect-timeout-nonpersistent-java-test-sync (testing (str "connection times out properly for non-persistent java sync " "request with short timeout") (let [request-options (-> "http://127.0.0.255:65535" (SimpleRequestOptions.) (.setConnectTimeoutMilliseconds 250)) time-before-connect (System/currentTimeMillis)] (is (wrapped-connect-exception-thrown? (Sync/get request-options)) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout")))) (deftest short-connect-timeout-persistent-java-test-sync (testing (str "connection times out properly for java persistent client sync " "request with short timeout") (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 250) (Sync/createClient))] (let [request-options (RequestOptions. "http://127.0.0.255:65535") time-before-connect (System/currentTimeMillis)] (is (wrapped-connect-exception-thrown? (.get client request-options)) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout"))))) (deftest short-connect-timeout-nonpersistent-clojure-test-sync (testing (str "connection times out properly for non-persistent clojure sync " "request with short timeout") (let [time-before-connect (System/currentTimeMillis)] (is (connect-exception-thrown? (sync/get "http://127.0.0.255:65535" {:connect-timeout-milliseconds 250})) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout")))) (deftest short-connect-timeout-persistent-clojure-test-sync (testing (str "connection times out properly for clojure persistent client " "sync request with short timeout") (with-open [client (sync/create-client {:connect-timeout-milliseconds 250})] (let [time-before-connect (System/currentTimeMillis)] (is (connect-exception-thrown? (common/get client "http://127.0.0.255:65535")) "Unexpected result for connection attempt") (is (elapsed-within-range? time-before-connect 2000) "Connection attempt took significantly longer than timeout"))))) (deftest longer-connect-timeout-test-sync (testing "connection succeeds for sync request with longer connect timeout" (testlogging/with-test-logging (testwebserver/with-test-webserver app port (let [url (str "http://localhost:" port "/hello")] (testing "java non-persistent sync client" (let [request-options (.. (SimpleRequestOptions. url) (setConnectTimeoutMilliseconds 2000)) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "java persistent sync client" (with-open [client (-> (ClientOptions.) (.setConnectTimeoutMilliseconds 2000) (Sync/createClient))] (let [response (.get client (RequestOptions. url))] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response))))))) (testing "clojure non-persistent sync client" (let [response (sync/get url {:as :text :connect-timeout-milliseconds 2000})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))) (testing "clojure persistent sync client" (with-open [client (sync/create-client {:connect-timeout-milliseconds 2000})] (let [response (common/get client url {:as :text})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))))))))) (deftest short-socket-timeout-nonpersistent-java-test-sync (testing (str "socket read times out properly for non-persistent java sync " "request with short timeout") (with-open [server (ServerSocket. 0)] (let [request-options (-> "http://127.0.0.1:" (str (.getLocalPort server)) (SimpleRequestOptions.) (.setSocketTimeoutMilliseconds 250)) time-before-connect (System/currentTimeMillis)] (is (wrapped-timeout-exception-thrown? (Sync/get request-options)) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest short-socket-timeout-persistent-java-test-sync (testing (str "socket read times out properly for persistent java sync " "request with short timeout") (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 250) (Sync/createClient)) server (ServerSocket. 0)] (let [request-options (-> "http://127.0.0.1:" (str (.getLocalPort server)) (RequestOptions.)) time-before-connect (System/currentTimeMillis)] (is (wrapped-timeout-exception-thrown? (.get client request-options)) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest short-socket-timeout-nonpersistent-clojure-test-sync (testing (str "socket read times out properly for non-persistent clojure " "sync request with short timeout") (with-open [server (ServerSocket. 0)] (let [url (str "http://127.0.0.1:" (.getLocalPort server)) time-before-connect (System/currentTimeMillis)] (is (thrown? SocketTimeoutException (sync/get url {:socket-timeout-milliseconds 250})) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest short-socket-timeout-persistent-clojure-test-sync (testing (str "socket read times out properly for clojure persistent client " "sync request with short timeout") (with-open [client (sync/create-client {:socket-timeout-milliseconds 250}) server (ServerSocket. 0)] (let [url (str "http://127.0.0.1:" (.getLocalPort server)) time-before-connect (System/currentTimeMillis)] (is (thrown? SocketTimeoutException (common/get client url)) "Unexpected result for get attempt") (is (elapsed-within-range? time-before-connect 2000) "Get attempt took significantly longer than timeout"))))) (deftest longer-socket-timeout-test-sync (testing "get succeeds for sync request with longer socket timeout" (testlogging/with-test-logging (testwebserver/with-test-webserver app port (let [url (str "http://localhost:" port "/hello")] (testing "java non-persistent sync client" (let [request-options (.. (SimpleRequestOptions. url) (setSocketTimeoutMilliseconds 2000)) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "java persistent sync client" (with-open [client (-> (ClientOptions.) (.setSocketTimeoutMilliseconds 2000) (Sync/createClient))] (let [response (.get client (RequestOptions. url))] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response))))))) (testing "clojure non-persistent sync client" (let [response (sync/get url {:as :text :socket-timeout-milliseconds 2000})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))) (testing "clojure persistent sync client" (with-open [client (sync/create-client {:socket-timeout-milliseconds 2000})] (let [response (common/get client url {:as :text})] (is (= 200 (:status response))) (is (= "Hello, World!" (:body response))))))))))) clj-http-client-0.9.0/test/puppetlabs/http/client/sync_ssl_test.clj000066400000000000000000000237401312026620400255040ustar00rootroot00000000000000(ns puppetlabs.http.client.sync-ssl-test (:import (com.puppetlabs.http.client Sync HttpClientException SimpleRequestOptions) (javax.net.ssl SSLHandshakeException) (java.net URI) (org.apache.http ConnectionClosedException)) (:require [clojure.test :refer :all] [puppetlabs.trapperkeeper.core :as tk] [puppetlabs.trapperkeeper.testutils.bootstrap :as testutils] [puppetlabs.trapperkeeper.testutils.logging :as testlogging] [puppetlabs.trapperkeeper.services.webserver.jetty9-service :as jetty9] [puppetlabs.http.client.sync :as sync] [schema.test :as schema-test])) (use-fixtures :once schema-test/validate-schemas) (defn app [req] {:status 200 :body "Hello, World!"}) (tk/defservice test-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler app "/hello") context)) (deftest sync-client-test-from-pems (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:ssl-host "0.0.0.0" :ssl-port 10080 :ssl-ca-cert "./dev-resources/ssl/ca.pem" :ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem"}} (testing "java sync client" (let [request-options (.. (SimpleRequestOptions. (URI. "https://localhost:10080/hello/")) (setSslCert "./dev-resources/ssl/cert.pem") (setSslKey "./dev-resources/ssl/key.pem") (setSslCaCert "./dev-resources/ssl/ca.pem")) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (sync/get "https://localhost:10080/hello/" {:ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem" :ssl-ca-cert "./dev-resources/ssl/ca.pem"})] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response))))))))) (deftest sync-client-test-from-ca-cert (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:ssl-host "0.0.0.0" :ssl-port 10080 :ssl-ca-cert "./dev-resources/ssl/ca.pem" :ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem" :client-auth "want"}} (testing "java sync client" (let [request-options (.. (SimpleRequestOptions. (URI. "https://localhost:10080/hello/")) (setSslCaCert "./dev-resources/ssl/ca.pem")) response (Sync/get request-options)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (sync/get "https://localhost:10080/hello/" {:ssl-ca-cert "./dev-resources/ssl/ca.pem"})] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response))))))))) (deftest sync-client-test-with-invalid-ca-cert (testlogging/with-test-logging (testutils/with-app-with-config app [jetty9/jetty9-service test-web-service] {:webserver {:ssl-host "0.0.0.0" :ssl-port 10081 :ssl-ca-cert "./dev-resources/ssl/ca.pem" :ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem" :client-auth "want"}} (testing "java sync client" (let [request-options (.. (SimpleRequestOptions. (URI. "https://localhost:10081/hello/")) (setSslCaCert "./dev-resources/ssl/alternate-ca.pem"))] (try (Sync/get request-options) ; fail if we don't get an exception (is (not true) "expected HttpClientException") (catch HttpClientException e (is (instance? SSLHandshakeException (.getCause e))))))) (testing "clojure sync client" (is (thrown? SSLHandshakeException (sync/get "https://localhost:10081/hello/" {:ssl-ca-cert "./dev-resources/ssl/alternate-ca.pem"}))))))) (defmacro with-server-with-protocols [server-protocols server-cipher-suites & body] `(testlogging/with-test-logging (testutils/with-app-with-config app# [jetty9/jetty9-service test-web-service] {:webserver (merge {:ssl-host "0.0.0.0" :ssl-port 10080 :ssl-ca-cert "./dev-resources/ssl/ca.pem" :ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem" :ssl-protocols ~server-protocols} (if ~server-cipher-suites {:cipher-suites ~server-cipher-suites}))} ~@body))) (defmacro java-unsupported-protocol-exception? [& body] `(try ~@body (catch HttpClientException e# (let [cause# (.getCause e#)] (or (and (instance? SSLHandshakeException cause#) (re-find #"not supported by the client" (.getMessage cause#))) (instance? ConnectionClosedException cause#)))))) (defn java-https-get-with-protocols [client-protocols client-cipher-suites] (let [request-options (.. (SimpleRequestOptions. (URI. "https://localhost:10080/hello/")) (setSslCert "./dev-resources/ssl/cert.pem") (setSslKey "./dev-resources/ssl/key.pem") (setSslCaCert "./dev-resources/ssl/ca.pem"))] (if client-protocols (.setSslProtocols request-options (into-array String client-protocols))) (if client-cipher-suites (.setSslCipherSuites request-options (into-array String client-cipher-suites))) (Sync/get request-options))) (defn clj-https-get-with-protocols [client-protocols client-cipher-suites] (let [ssl-opts (merge {:ssl-cert "./dev-resources/ssl/cert.pem" :ssl-key "./dev-resources/ssl/key.pem" :ssl-ca-cert "./dev-resources/ssl/ca.pem"} (if client-protocols {:ssl-protocols client-protocols}) (if client-cipher-suites {:cipher-suites client-cipher-suites}))] (sync/get "https://localhost:10080/hello/" ssl-opts))) (deftest sync-client-test-ssl-protocols (testing "should be able to connect to a TLSv1.2 server by default" (with-server-with-protocols ["TLSv1.2"] nil (testing "java sync client" (let [response (java-https-get-with-protocols nil nil)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (clj-https-get-with-protocols nil nil)] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))))) (testing "should be able to connect to a server with non-default protocol if configured" (with-server-with-protocols ["SSLv3"] nil (testing "java sync client" (let [response (java-https-get-with-protocols ["SSLv3"] nil)] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (clj-https-get-with-protocols ["SSLv3"] nil)] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))))) (testing "should not connect to an SSLv3 server by default" (with-server-with-protocols ["SSLv3"] nil (testing "java sync client" (is (java-unsupported-protocol-exception? (java-https-get-with-protocols nil nil)))) (testing "clojure sync client" (is (thrown-with-msg? SSLHandshakeException #"not supported by the client" (clj-https-get-with-protocols nil nil)))))) (testing "should not connect to a server when protocols don't overlap" (with-server-with-protocols ["TLSv1.1"] nil (testing "java sync client" (is (java-unsupported-protocol-exception? (java-https-get-with-protocols ["TLSv1.2"] nil)))) (testing "clojure sync client" (is (thrown-with-msg? SSLHandshakeException #"not supported by the client" (clj-https-get-with-protocols ["TLSv1.2"] nil))))))) (deftest sync-client-test-cipher-suites (testing "should not connect to a server with no overlapping cipher suites" (with-server-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_SHA"] (testing "java sync client" (is (java-unsupported-protocol-exception? (java-https-get-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_MD5"])))) (testing "clojure sync client" (is (thrown? ConnectionClosedException (clj-https-get-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_MD5"])))))) (testing "should connect to a server with overlapping cipher suites" (with-server-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_MD5"] (testing "java sync client" (let [response (java-https-get-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_MD5"])] (is (= 200 (.getStatus response))) (is (= "Hello, World!" (slurp (.getBody response)))))) (testing "clojure sync client" (let [response (clj-https-get-with-protocols ["SSLv3"] ["SSL_RSA_WITH_RC4_128_MD5"])] (is (= 200 (:status response))) (is (= "Hello, World!" (slurp (:body response)))))))))clj-http-client-0.9.0/test/puppetlabs/http/client/test_common.clj000066400000000000000000000032331312026620400251320ustar00rootroot00000000000000(ns puppetlabs.http.client.test-common (:require [ring.middleware.params :as ring-params] [puppetlabs.trapperkeeper.core :as tk] [puppetlabs.trapperkeeper.testutils.logging :as testlogging]) (:import (java.net ConnectException SocketTimeoutException))) (defn query-params-test [req] {:status 200 :body (str (:query-params req))}) (def app-wrapped (ring-params/wrap-params query-params-test)) (tk/defservice test-params-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler app-wrapped "/params") context)) (def queryparams {"foo" "bar" "baz" "lux"}) (def query-options {:method :get :url "http://localhost:8080/params/" :query-params queryparams :as :text}) (defn redirect-test-handler [req] (condp = (:uri req) "/hello/world" {:status 200 :body "Hello, World!"} "/hello" {:status 302 :headers {"Location" "/hello/world"} :body ""} {:status 404 :body "D'oh"})) (tk/defservice redirect-web-service [[:WebserverService add-ring-handler]] (init [this context] (add-ring-handler redirect-test-handler "/hello") context)) (defmacro connect-exception-thrown? [& body] `(try (testlogging/with-test-logging ~@body) (catch SocketTimeoutException _# true) (catch ConnectException _# true))) (defn elapsed-within-range? [start-time-milliseconds duration-milliseconds] (<= (System/currentTimeMillis) (+ start-time-milliseconds duration-milliseconds)))