pax_global_header00006660000000000000000000000064131544341100014506gustar00rootroot0000000000000052 comment=16f33bf76f06afb58307a652ceb48f597e48a095 node-fetch-1.7.3/000077500000000000000000000000001315443411000135325ustar00rootroot00000000000000node-fetch-1.7.3/.gitignore000066400000000000000000000013531315443411000155240ustar00rootroot00000000000000# Logs logs *.log # Runtime data pids *.pid *.seed # Directory for instrumented libs generated by jscoverage/JSCover lib-cov # Coverage directory used by tools like istanbul coverage # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) .grunt # Compiled binary addons (http://nodejs.org/api/addons.html) build/Release # Dependency directory # Commenting this out is preferred by some people, see # https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git- node_modules # Users Environment Variables .lock-wscript # OS files .DS_Store # Coveralls token files .coveralls.yml ## ignore some files from 2.x branch .nyc_output lib/index.js lib/index.es.js package-lock.json node-fetch-1.7.3/.travis.yml000066400000000000000000000004741315443411000156500ustar00rootroot00000000000000language: node_js node_js: - "0.10" - "0.12" - "node" env: - FORMDATA_VERSION=1.0.0 - FORMDATA_VERSION=2.1.0 before_script: - 'if [ "$FORMDATA_VERSION" ]; then npm install form-data@^$FORMDATA_VERSION; fi' before_install: if [[ `npm -v` < 3 ]]; then npm install -g npm@1.4.28; fi script: npm run coveragenode-fetch-1.7.3/CHANGELOG.md000066400000000000000000000122371315443411000153500ustar00rootroot00000000000000 Changelog ========= # 1.x release (Note: `1.x` will only have backported bugfix releases beyond `1.7.0`) ## v1.7.3 - Enhance: `FetchError` now gives a correct trace stack (backport from v2.x relese). ## v1.7.2 - Fix: when using node-fetch with test framework such as `jest`, `instanceof` check could fail in `Headers` class. This is causing some header values, such as `set-cookie`, to be dropped incorrectly. ## v1.7.1 - Fix: close local test server properly under Node 8. ## v1.7.0 - Fix: revert change in `v1.6.2` where 204 no-content response is handled with a special case, this conflicts with browser Fetch implementation (as browsers always throw when res.json() parses an empty string). Since this is an operational error, it's wrapped in a `FetchError` for easier error handling. - Fix: move code coverage tool to codecov platform and update travis config ## v1.6.3 - Enhance: error handling document to explain `FetchError` design - Fix: support `form-data` 2.x releases (requires `form-data` >= 2.1.0) ## v1.6.2 - Enhance: minor document update - Fix: response.json() returns empty object on 204 no-content response instead of throwing a syntax error ## v1.6.1 - Fix: if `res.body` is a non-stream non-formdata object, we will call `body.toString` and send it as a string - Fix: `counter` value is incorrectly set to `follow` value when wrapping Request instance - Fix: documentation update ## v1.6.0 - Enhance: added `res.buffer()` api for convenience, it returns body as a Node.js buffer - Enhance: better old server support by handling raw deflate response - Enhance: skip encoding detection for non-HTML/XML response - Enhance: minor document update - Fix: HEAD request doesn't need decompression, as body is empty - Fix: `req.body` now accepts a Node.js buffer ## v1.5.3 - Fix: handle 204 and 304 responses when body is empty but content-encoding is gzip/deflate - Fix: allow resolving response and cloned response in any order - Fix: avoid setting `content-length` when `form-data` body use streams - Fix: send DELETE request with content-length when body is present - Fix: allow any url when calling new Request, but still reject non-http(s) url in fetch ## v1.5.2 - Fix: allow node.js core to handle keep-alive connection pool when passing a custom agent ## v1.5.1 - Fix: redirect mode `manual` should work even when there is no redirection or broken redirection ## v1.5.0 - Enhance: rejected promise now use custom `Error` (thx to @pekeler) - Enhance: `FetchError` contains `err.type` and `err.code`, allows for better error handling (thx to @pekeler) - Enhance: basic support for redirect mode `manual` and `error`, allows for location header extraction (thx to @jimmywarting for the initial PR) ## v1.4.1 - Fix: wrapping Request instance with FormData body again should preserve the body as-is ## v1.4.0 - Enhance: Request and Response now have `clone` method (thx to @kirill-konshin for the initial PR) - Enhance: Request and Response now have proper string and buffer body support (thx to @kirill-konshin) - Enhance: Body constructor has been refactored out (thx to @kirill-konshin) - Enhance: Headers now has `forEach` method (thx to @tricoder42) - Enhance: back to 100% code coverage - Fix: better form-data support (thx to @item4) - Fix: better character encoding detection under chunked encoding (thx to @dsuket for the initial PR) ## v1.3.3 - Fix: make sure `Content-Length` header is set when body is string for POST/PUT/PATCH requests - Fix: handle body stream error, for cases such as incorrect `Content-Encoding` header - Fix: when following certain redirects, use `GET` on subsequent request per Fetch Spec - Fix: `Request` and `Response` constructors now parse headers input using `Headers` ## v1.3.2 - Enhance: allow auto detect of form-data input (no `FormData` spec on node.js, this is form-data specific feature) ## v1.3.1 - Enhance: allow custom host header to be set (server-side only feature, as it's a forbidden header on client-side) ## v1.3.0 - Enhance: now `fetch.Request` is exposed as well ## v1.2.1 - Enhance: `Headers` now normalized `Number` value to `String`, prevent common mistakes ## v1.2.0 - Enhance: now fetch.Headers and fetch.Response are exposed, making testing easier ## v1.1.2 - Fix: `Headers` should only support `String` and `Array` properties, and ignore others ## v1.1.1 - Enhance: now req.headers accept both plain object and `Headers` instance ## v1.1.0 - Enhance: timeout now also applies to response body (in case of slow response) - Fix: timeout is now cleared properly when fetch is done/has failed ## v1.0.6 - Fix: less greedy content-type charset matching ## v1.0.5 - Fix: when `follow = 0`, fetch should not follow redirect - Enhance: update tests for better coverage - Enhance: code formatting - Enhance: clean up doc ## v1.0.4 - Enhance: test iojs support - Enhance: timeout attached to socket event only fire once per redirect ## v1.0.3 - Fix: response size limit should reject large chunk - Enhance: added character encoding detection for xml, such as rss/atom feed (encoding in DTD) ## v1.0.2 - Fix: added res.ok per spec change ## v1.0.0 - Enhance: better test coverage and doc # 0.x release ## v0.1 - Major: initial public release node-fetch-1.7.3/ERROR-HANDLING.md000066400000000000000000000023201315443411000161240ustar00rootroot00000000000000 Error handling with node-fetch ============================== Because `window.fetch` isn't designed to transparent about the cause of request errors, we have to come up with our own solutions. The basics: - All [operational errors](https://www.joyent.com/node-js/production/design/errors) are rejected as [FetchError](https://github.com/bitinn/node-fetch/blob/master/lib/fetch-error.js), you can handle them all through promise `catch` clause. - All errors comes with `err.message` detailing the cause of errors. - All errors originated from `node-fetch` are marked with custom `err.type`. - All errors originated from Node.js core are marked with `err.type = system`, and contains addition `err.code` and `err.errno` for error handling, they are alias to error codes thrown by Node.js core. - [Programmer errors](https://www.joyent.com/node-js/production/design/errors) are either thrown as soon as possible, or rejected with default `Error` with `err.message` for ease of troubleshooting. List of error types: - Because we maintain 100% coverage, see [test.js](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for a full list of custom `FetchError` types, as well as some of the common errors from Node.js node-fetch-1.7.3/LICENSE.md000066400000000000000000000020671315443411000151430ustar00rootroot00000000000000The MIT License (MIT) Copyright (c) 2016 David Frank Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. node-fetch-1.7.3/LIMITS.md000066400000000000000000000024751315443411000150650ustar00rootroot00000000000000 Known differences ================= *As of 1.x release* - Topics such as Cross-Origin, Content Security Policy, Mixed Content, Service Workers are ignored, given our server-side context. - URL input must be an absolute URL, using either `http` or `https` as scheme. - On the upside, there are no forbidden headers, and `res.url` contains the final url when following redirects. - For convenience, `res.body` is a transform stream, so decoding can be handled independently. - Similarly, `req.body` can either be a string, a buffer or a readable stream. - Also, you can handle rejected fetch requests through checking `err.type` and `err.code`. - Only support `res.text()`, `res.json()`, `res.buffer()` at the moment, until there are good use-cases for blob/arrayBuffer. - There is currently no built-in caching, as server-side caching varies by use-cases. - Current implementation lacks server-side cookie store, you will need to extract `Set-Cookie` headers manually. - If you are using `res.clone()` and writing an isomorphic app, note that stream on Node.js have a smaller internal buffer size (16Kb, aka `highWaterMark`) from client-side browsers (>1Mb, not consistent across browsers). - ES6 features such as `headers.entries()` are missing at the moment, but you can use `headers.raw()` to retrieve the raw headers object. node-fetch-1.7.3/README.md000066400000000000000000000140371315443411000150160ustar00rootroot00000000000000 node-fetch ========== [![npm version][npm-image]][npm-url] [![build status][travis-image]][travis-url] [![coverage status][codecov-image]][codecov-url] A light-weight module that brings `window.fetch` to Node.js # Motivation Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `Fetch` API directly? Hence `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime. See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side). # Features - Stay consistent with `window.fetch` API. - Make conscious trade-off when following [whatwg fetch spec](https://fetch.spec.whatwg.org/) and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known difference. - Use native promise, but allow substituting it with [insert your favorite promise library]. - Use native stream for body, on both request and response. - Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically. - Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md) for troubleshooting. # Difference from client-side fetch - See [Known Differences](https://github.com/bitinn/node-fetch/blob/master/LIMITS.md) for details. - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue. - Pull requests are welcomed too! # Install `npm install node-fetch --save` # Usage ```javascript var fetch = require('node-fetch'); // if you are on node v0.10, set a Promise library first, eg. // fetch.Promise = require('bluebird'); // plain text or html fetch('https://github.com/') .then(function(res) { return res.text(); }).then(function(body) { console.log(body); }); // json fetch('https://api.github.com/users/github') .then(function(res) { return res.json(); }).then(function(json) { console.log(json); }); // catching network error // 3xx-5xx responses are NOT network errors, and should be handled in then() // you only need one catch() at the end of your promise chain fetch('http://domain.invalid/') .catch(function(err) { console.log(err); }); // stream // the node.js way is to use stream when possible fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') .then(function(res) { var dest = fs.createWriteStream('./octocat.png'); res.body.pipe(dest); }); // buffer // if you prefer to cache binary data in full, use buffer() // note that buffer() is a node-fetch only API var fileType = require('file-type'); fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') .then(function(res) { return res.buffer(); }).then(function(buffer) { fileType(buffer); }); // meta fetch('https://github.com/') .then(function(res) { console.log(res.ok); console.log(res.status); console.log(res.statusText); console.log(res.headers.raw()); console.log(res.headers.get('content-type')); }); // post fetch('http://httpbin.org/post', { method: 'POST', body: 'a=1' }) .then(function(res) { return res.json(); }).then(function(json) { console.log(json); }); // post with stream from resumer var resumer = require('resumer'); var stream = resumer().queue('a=1').end(); fetch('http://httpbin.org/post', { method: 'POST', body: stream }) .then(function(res) { return res.json(); }).then(function(json) { console.log(json); }); // post with form-data (detect multipart) var FormData = require('form-data'); var form = new FormData(); form.append('a', 1); fetch('http://httpbin.org/post', { method: 'POST', body: form }) .then(function(res) { return res.json(); }).then(function(json) { console.log(json); }); // post with form-data (custom headers) // note that getHeaders() is non-standard API var FormData = require('form-data'); var form = new FormData(); form.append('a', 1); fetch('http://httpbin.org/post', { method: 'POST', body: form, headers: form.getHeaders() }) .then(function(res) { return res.json(); }).then(function(json) { console.log(json); }); // node 0.12+, yield with co var co = require('co'); co(function *() { var res = yield fetch('https://api.github.com/users/github'); var json = yield res.json(); console.log(res); }); ``` See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples. # API ## fetch(url, options) Returns a `Promise` ### Url Should be an absolute url, eg `http://example.com/` ### Options default values are shown, note that only `method`, `headers`, `redirect` and `body` are allowed in `window.fetch`, others are node.js extensions. ``` { method: 'GET' , headers: {} // request header. format {a:'1'} or {b:['1','2','3']} , redirect: 'follow' // set to `manual` to extract redirect headers, `error` to reject redirect , follow: 20 // maximum redirect count. 0 to not follow redirect , timeout: 0 // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies) , compress: true // support gzip/deflate content encoding. false to disable , size: 0 // maximum response body size in bytes. 0 to disable , body: empty // request body. can be a string, buffer, readable stream , agent: null // http.Agent instance, allows custom proxy, certificate etc. } ``` # License MIT # Acknowledgement Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference. [npm-image]: https://img.shields.io/npm/v/node-fetch.svg?style=flat-square [npm-url]: https://www.npmjs.com/package/node-fetch [travis-image]: https://img.shields.io/travis/bitinn/node-fetch.svg?style=flat-square [travis-url]: https://travis-ci.org/bitinn/node-fetch [codecov-image]: https://img.shields.io/codecov/c/github/bitinn/node-fetch.svg?style=flat-square [codecov-url]: https://codecov.io/gh/bitinn/node-fetch node-fetch-1.7.3/index.js000066400000000000000000000176641315443411000152150ustar00rootroot00000000000000 /** * index.js * * a request API compatible with window.fetch */ var parse_url = require('url').parse; var resolve_url = require('url').resolve; var http = require('http'); var https = require('https'); var zlib = require('zlib'); var stream = require('stream'); var Body = require('./lib/body'); var Response = require('./lib/response'); var Headers = require('./lib/headers'); var Request = require('./lib/request'); var FetchError = require('./lib/fetch-error'); // commonjs module.exports = Fetch; // es6 default export compatibility module.exports.default = module.exports; /** * Fetch class * * @param Mixed url Absolute url or Request instance * @param Object opts Fetch options * @return Promise */ function Fetch(url, opts) { // allow call as function if (!(this instanceof Fetch)) return new Fetch(url, opts); // allow custom promise if (!Fetch.Promise) { throw new Error('native promise missing, set Fetch.Promise to your favorite alternative'); } Body.Promise = Fetch.Promise; var self = this; // wrap http.request into fetch return new Fetch.Promise(function(resolve, reject) { // build request object var options = new Request(url, opts); if (!options.protocol || !options.hostname) { throw new Error('only absolute urls are supported'); } if (options.protocol !== 'http:' && options.protocol !== 'https:') { throw new Error('only http(s) protocols are supported'); } var send; if (options.protocol === 'https:') { send = https.request; } else { send = http.request; } // normalize headers var headers = new Headers(options.headers); if (options.compress) { headers.set('accept-encoding', 'gzip,deflate'); } if (!headers.has('user-agent')) { headers.set('user-agent', 'node-fetch/1.0 (+https://github.com/bitinn/node-fetch)'); } if (!headers.has('connection') && !options.agent) { headers.set('connection', 'close'); } if (!headers.has('accept')) { headers.set('accept', '*/*'); } // detect form data input from form-data module, this hack avoid the need to pass multipart header manually if (!headers.has('content-type') && options.body && typeof options.body.getBoundary === 'function') { headers.set('content-type', 'multipart/form-data; boundary=' + options.body.getBoundary()); } // bring node-fetch closer to browser behavior by setting content-length automatically if (!headers.has('content-length') && /post|put|patch|delete/i.test(options.method)) { if (typeof options.body === 'string') { headers.set('content-length', Buffer.byteLength(options.body)); // detect form data input from form-data module, this hack avoid the need to add content-length header manually } else if (options.body && typeof options.body.getLengthSync === 'function') { // for form-data 1.x if (options.body._lengthRetrievers && options.body._lengthRetrievers.length == 0) { headers.set('content-length', options.body.getLengthSync().toString()); // for form-data 2.x } else if (options.body.hasKnownLength && options.body.hasKnownLength()) { headers.set('content-length', options.body.getLengthSync().toString()); } // this is only necessary for older nodejs releases (before iojs merge) } else if (options.body === undefined || options.body === null) { headers.set('content-length', '0'); } } options.headers = headers.raw(); // http.request only support string as host header, this hack make custom host header possible if (options.headers.host) { options.headers.host = options.headers.host[0]; } // send request var req = send(options); var reqTimeout; if (options.timeout) { req.once('socket', function(socket) { reqTimeout = setTimeout(function() { req.abort(); reject(new FetchError('network timeout at: ' + options.url, 'request-timeout')); }, options.timeout); }); } req.on('error', function(err) { clearTimeout(reqTimeout); reject(new FetchError('request to ' + options.url + ' failed, reason: ' + err.message, 'system', err)); }); req.on('response', function(res) { clearTimeout(reqTimeout); // handle redirect if (self.isRedirect(res.statusCode) && options.redirect !== 'manual') { if (options.redirect === 'error') { reject(new FetchError('redirect mode is set to error: ' + options.url, 'no-redirect')); return; } if (options.counter >= options.follow) { reject(new FetchError('maximum redirect reached at: ' + options.url, 'max-redirect')); return; } if (!res.headers.location) { reject(new FetchError('redirect location header missing at: ' + options.url, 'invalid-redirect')); return; } // per fetch spec, for POST request with 301/302 response, or any request with 303 response, use GET when following redirect if (res.statusCode === 303 || ((res.statusCode === 301 || res.statusCode === 302) && options.method === 'POST')) { options.method = 'GET'; delete options.body; delete options.headers['content-length']; } options.counter++; resolve(Fetch(resolve_url(options.url, res.headers.location), options)); return; } // normalize location header for manual redirect mode var headers = new Headers(res.headers); if (options.redirect === 'manual' && headers.has('location')) { headers.set('location', resolve_url(options.url, headers.get('location'))); } // prepare response var body = res.pipe(new stream.PassThrough()); var response_options = { url: options.url , status: res.statusCode , statusText: res.statusMessage , headers: headers , size: options.size , timeout: options.timeout }; // response object var output; // in following scenarios we ignore compression support // 1. compression support is disabled // 2. HEAD request // 3. no content-encoding header // 4. no content response (204) // 5. content not modified response (304) if (!options.compress || options.method === 'HEAD' || !headers.has('content-encoding') || res.statusCode === 204 || res.statusCode === 304) { output = new Response(body, response_options); resolve(output); return; } // otherwise, check for gzip or deflate var name = headers.get('content-encoding'); // for gzip if (name == 'gzip' || name == 'x-gzip') { body = body.pipe(zlib.createGunzip()); output = new Response(body, response_options); resolve(output); return; // for deflate } else if (name == 'deflate' || name == 'x-deflate') { // handle the infamous raw deflate response from old servers // a hack for old IIS and Apache servers var raw = res.pipe(new stream.PassThrough()); raw.once('data', function(chunk) { // see http://stackoverflow.com/questions/37519828 if ((chunk[0] & 0x0F) === 0x08) { body = body.pipe(zlib.createInflate()); } else { body = body.pipe(zlib.createInflateRaw()); } output = new Response(body, response_options); resolve(output); }); return; } // otherwise, use response as-is output = new Response(body, response_options); resolve(output); return; }); // accept string, buffer or readable stream as body // per spec we will call tostring on non-stream objects if (typeof options.body === 'string') { req.write(options.body); req.end(); } else if (options.body instanceof Buffer) { req.write(options.body); req.end(); } else if (typeof options.body === 'object' && options.body.pipe) { options.body.pipe(req); } else if (typeof options.body === 'object') { req.write(options.body.toString()); req.end(); } else { req.end(); } }); }; /** * Redirect code matching * * @param Number code Status code * @return Boolean */ Fetch.prototype.isRedirect = function(code) { return code === 301 || code === 302 || code === 303 || code === 307 || code === 308; } // expose Promise Fetch.Promise = global.Promise; Fetch.Response = Response; Fetch.Headers = Headers; Fetch.Request = Request; node-fetch-1.7.3/lib/000077500000000000000000000000001315443411000143005ustar00rootroot00000000000000node-fetch-1.7.3/lib/body.js000066400000000000000000000126001315443411000155720ustar00rootroot00000000000000 /** * body.js * * Body interface provides common methods for Request and Response */ var convert = require('encoding').convert; var bodyStream = require('is-stream'); var PassThrough = require('stream').PassThrough; var FetchError = require('./fetch-error'); module.exports = Body; /** * Body class * * @param Stream body Readable stream * @param Object opts Response options * @return Void */ function Body(body, opts) { opts = opts || {}; this.body = body; this.bodyUsed = false; this.size = opts.size || 0; this.timeout = opts.timeout || 0; this._raw = []; this._abort = false; } /** * Decode response as json * * @return Promise */ Body.prototype.json = function() { var self = this; return this._decode().then(function(buffer) { try { return JSON.parse(buffer.toString()); } catch (err) { return Body.Promise.reject(new FetchError('invalid json response body at ' + self.url + ' reason: ' + err.message, 'invalid-json')); } }); }; /** * Decode response as text * * @return Promise */ Body.prototype.text = function() { return this._decode().then(function(buffer) { return buffer.toString(); }); }; /** * Decode response as buffer (non-spec api) * * @return Promise */ Body.prototype.buffer = function() { return this._decode(); }; /** * Decode buffers into utf-8 string * * @return Promise */ Body.prototype._decode = function() { var self = this; if (this.bodyUsed) { return Body.Promise.reject(new Error('body used already for: ' + this.url)); } this.bodyUsed = true; this._bytes = 0; this._abort = false; this._raw = []; return new Body.Promise(function(resolve, reject) { var resTimeout; // body is string if (typeof self.body === 'string') { self._bytes = self.body.length; self._raw = [new Buffer(self.body)]; return resolve(self._convert()); } // body is buffer if (self.body instanceof Buffer) { self._bytes = self.body.length; self._raw = [self.body]; return resolve(self._convert()); } // allow timeout on slow response body if (self.timeout) { resTimeout = setTimeout(function() { self._abort = true; reject(new FetchError('response timeout at ' + self.url + ' over limit: ' + self.timeout, 'body-timeout')); }, self.timeout); } // handle stream error, such as incorrect content-encoding self.body.on('error', function(err) { reject(new FetchError('invalid response body at: ' + self.url + ' reason: ' + err.message, 'system', err)); }); // body is stream self.body.on('data', function(chunk) { if (self._abort || chunk === null) { return; } if (self.size && self._bytes + chunk.length > self.size) { self._abort = true; reject(new FetchError('content size at ' + self.url + ' over limit: ' + self.size, 'max-size')); return; } self._bytes += chunk.length; self._raw.push(chunk); }); self.body.on('end', function() { if (self._abort) { return; } clearTimeout(resTimeout); resolve(self._convert()); }); }); }; /** * Detect buffer encoding and convert to target encoding * ref: http://www.w3.org/TR/2011/WD-html5-20110113/parsing.html#determining-the-character-encoding * * @param String encoding Target encoding * @return String */ Body.prototype._convert = function(encoding) { encoding = encoding || 'utf-8'; var ct = this.headers.get('content-type'); var charset = 'utf-8'; var res, str; // header if (ct) { // skip encoding detection altogether if not html/xml/plain text if (!/text\/html|text\/plain|\+xml|\/xml/i.test(ct)) { return Buffer.concat(this._raw); } res = /charset=([^;]*)/i.exec(ct); } // no charset in content type, peek at response body for at most 1024 bytes if (!res && this._raw.length > 0) { for (var i = 0; i < this._raw.length; i++) { str += this._raw[i].toString() if (str.length > 1024) { break; } } str = str.substr(0, 1024); } // html5 if (!res && str) { res = /= 200 && this.status < 300; Body.call(this, body, opts); } Response.prototype = Object.create(Body.prototype); /** * Clone this response * * @return Response */ Response.prototype.clone = function() { return new Response(this._clone(this), { url: this.url , status: this.status , statusText: this.statusText , headers: this.headers , ok: this.ok }); }; node-fetch-1.7.3/package.json000066400000000000000000000020461315443411000160220ustar00rootroot00000000000000{ "name": "node-fetch", "version": "1.7.3", "description": "A light-weight module that brings window.fetch to node.js and io.js", "main": "index.js", "scripts": { "test": "mocha test/test.js", "report": "istanbul cover _mocha -- -R spec test/test.js", "coverage": "istanbul cover _mocha --report lcovonly -- -R spec test/test.js && codecov" }, "repository": { "type": "git", "url": "https://github.com/bitinn/node-fetch.git" }, "keywords": [ "fetch", "http", "promise" ], "author": "David Frank", "license": "MIT", "bugs": { "url": "https://github.com/bitinn/node-fetch/issues" }, "homepage": "https://github.com/bitinn/node-fetch", "devDependencies": { "bluebird": "^3.3.4", "chai": "^3.5.0", "chai-as-promised": "^5.2.0", "codecov": "^1.0.1", "form-data": ">=1.0.0", "istanbul": "^0.4.2", "mocha": "^2.1.0", "parted": "^0.1.1", "promise": "^7.1.1", "resumer": "0.0.0" }, "dependencies": { "encoding": "^0.1.11", "is-stream": "^1.0.1" } } node-fetch-1.7.3/test/000077500000000000000000000000001315443411000145115ustar00rootroot00000000000000node-fetch-1.7.3/test/dummy.txt000066400000000000000000000000141315443411000164000ustar00rootroot00000000000000i am a dummynode-fetch-1.7.3/test/server.js000066400000000000000000000172241315443411000163630ustar00rootroot00000000000000 var http = require('http'); var parse = require('url').parse; var zlib = require('zlib'); var stream = require('stream'); var convert = require('encoding').convert; var Multipart = require('parted').multipart; module.exports = TestServer; function TestServer() { this.server = http.createServer(this.router); this.port = 30001; this.hostname = 'localhost'; // node 8 default keepalive timeout is 5000ms // make it shorter here as we want to close server quickly at the end of tests this.server.keepAliveTimeout = 1000; this.server.on('error', function(err) { console.log(err.stack); }); this.server.on('connection', function(socket) { socket.setTimeout(1500); }); } TestServer.prototype.start = function(cb) { this.server.listen(this.port, this.hostname, cb); } TestServer.prototype.stop = function(cb) { this.server.close(cb); } TestServer.prototype.router = function(req, res) { var p = parse(req.url).pathname; if (p === '/hello') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.end('world'); } if (p === '/plain') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.end('text'); } if (p === '/options') { res.statusCode = 200; res.setHeader('Allow', 'GET, HEAD, OPTIONS'); res.end('hello world'); } if (p === '/html') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html'); res.end(''); } if (p === '/json') { res.statusCode = 200; res.setHeader('Content-Type', 'application/json'); res.end(JSON.stringify({ name: 'value' })); } if (p === '/gzip') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Encoding', 'gzip'); zlib.gzip('hello world', function(err, buffer) { res.end(buffer); }); } if (p === '/deflate') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Encoding', 'deflate'); zlib.deflate('hello world', function(err, buffer) { res.end(buffer); }); } if (p === '/deflate-raw') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Encoding', 'deflate'); zlib.deflateRaw('hello world', function(err, buffer) { res.end(buffer); }); } if (p === '/sdch') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Encoding', 'sdch'); res.end('fake sdch string'); } if (p === '/invalid-content-encoding') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.setHeader('Content-Encoding', 'gzip'); res.end('fake gzip string'); } if (p === '/timeout') { setTimeout(function() { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.end('text'); }, 1000); } if (p === '/slow') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.write('test'); setTimeout(function() { res.end('test'); }, 1000); } if (p === '/cookie') { res.statusCode = 200; res.setHeader('Set-Cookie', ['a=1', 'b=1']); res.end('cookie'); } if (p === '/size/chunk') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); setTimeout(function() { res.write('test'); }, 50); setTimeout(function() { res.end('test'); }, 100); } if (p === '/size/long') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain'); res.end('testtest'); } if (p === '/encoding/gbk') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html'); res.end(convert('
中文
', 'gbk')); } if (p === '/encoding/gb2312') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html'); res.end(convert('
中文
', 'gb2312')); } if (p === '/encoding/shift-jis') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html; charset=Shift-JIS'); res.end(convert('
日本語
', 'Shift_JIS')); } if (p === '/encoding/euc-jp') { res.statusCode = 200; res.setHeader('Content-Type', 'text/xml'); res.end(convert('日本語', 'EUC-JP')); } if (p === '/encoding/utf8') { res.statusCode = 200; res.end('中文'); } if (p === '/encoding/order1') { res.statusCode = 200; res.setHeader('Content-Type', 'charset=gbk; text/plain'); res.end(convert('中文', 'gbk')); } if (p === '/encoding/order2') { res.statusCode = 200; res.setHeader('Content-Type', 'text/plain; charset=gbk; qs=1'); res.end(convert('中文', 'gbk')); } if (p === '/encoding/chunked') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html'); res.setHeader('Transfer-Encoding', 'chunked'); var padding = 'a'; for (var i = 0; i < 10; i++) { res.write(padding); } res.end(convert('
日本語
', 'Shift_JIS')); } if (p === '/encoding/invalid') { res.statusCode = 200; res.setHeader('Content-Type', 'text/html'); res.setHeader('Transfer-Encoding', 'chunked'); // because node v0.12 doesn't have str.repeat var padding = new Array(120 + 1).join('a'); for (var i = 0; i < 10; i++) { res.write(padding); } res.end(convert('中文', 'gbk')); } if (p === '/redirect/301') { res.statusCode = 301; res.setHeader('Location', '/inspect'); res.end(); } if (p === '/redirect/302') { res.statusCode = 302; res.setHeader('Location', '/inspect'); res.end(); } if (p === '/redirect/303') { res.statusCode = 303; res.setHeader('Location', '/inspect'); res.end(); } if (p === '/redirect/307') { res.statusCode = 307; res.setHeader('Location', '/inspect'); res.end(); } if (p === '/redirect/308') { res.statusCode = 308; res.setHeader('Location', '/inspect'); res.end(); } if (p === '/redirect/chain') { res.statusCode = 301; res.setHeader('Location', '/redirect/301'); res.end(); } if (p === '/error/redirect') { res.statusCode = 301; //res.setHeader('Location', '/inspect'); res.end(); } if (p === '/error/400') { res.statusCode = 400; res.setHeader('Content-Type', 'text/plain'); res.end('client error'); } if (p === '/error/404') { res.statusCode = 404; res.setHeader('Content-Encoding', 'gzip'); res.end(); } if (p === '/error/500') { res.statusCode = 500; res.setHeader('Content-Type', 'text/plain'); res.end('server error'); } if (p === '/error/reset') { res.destroy(); } if (p === '/error/json') { res.statusCode = 200; res.setHeader('Content-Type', 'application/json'); res.end('invalid json'); } if (p === '/no-content') { res.statusCode = 204; res.end(); } if (p === '/no-content/gzip') { res.statusCode = 204; res.setHeader('Content-Encoding', 'gzip'); res.end(); } if (p === '/not-modified') { res.statusCode = 304; res.end(); } if (p === '/not-modified/gzip') { res.statusCode = 304; res.setHeader('Content-Encoding', 'gzip'); res.end(); } if (p === '/inspect') { res.statusCode = 200; res.setHeader('Content-Type', 'application/json'); var body = ''; req.on('data', function(c) { body += c }); req.on('end', function() { res.end(JSON.stringify({ method: req.method, url: req.url, headers: req.headers, body: body })); }); } if (p === '/multipart') { res.statusCode = 200; res.setHeader('Content-Type', 'application/json'); var parser = new Multipart(req.headers['content-type']); var body = ''; parser.on('part', function(field, part) { body += field + '=' + part; }); parser.on('end', function() { res.end(JSON.stringify({ method: req.method, url: req.url, headers: req.headers, body: body })); }); req.pipe(parser); } } node-fetch-1.7.3/test/test.js000066400000000000000000001223031315443411000160270ustar00rootroot00000000000000 // test tools var chai = require('chai'); var cap = require('chai-as-promised'); chai.use(cap); var expect = chai.expect; var bluebird = require('bluebird'); var then = require('promise'); var spawn = require('child_process').spawn; var stream = require('stream'); var resumer = require('resumer'); var FormData = require('form-data'); var http = require('http'); var fs = require('fs'); var TestServer = require('./server'); // test subjects var fetch = require('../index.js'); var Headers = require('../lib/headers.js'); var Response = require('../lib/response.js'); var Request = require('../lib/request.js'); var Body = require('../lib/body.js'); var FetchError = require('../lib/fetch-error.js'); // test with native promise on node 0.11, and bluebird for node 0.10 fetch.Promise = fetch.Promise || bluebird; var url, opts, local, base; describe('node-fetch', function() { before(function(done) { local = new TestServer(); base = 'http://' + local.hostname + ':' + local.port; local.start(done); }); after(function(done) { local.stop(done); }); it('should return a promise', function() { url = 'http://example.com/'; var p = fetch(url); expect(p).to.be.an.instanceof(fetch.Promise); expect(p).to.have.property('then'); }); it('should allow custom promise', function() { url = 'http://example.com/'; var old = fetch.Promise; fetch.Promise = then; expect(fetch(url)).to.be.an.instanceof(then); expect(fetch(url)).to.not.be.an.instanceof(bluebird); fetch.Promise = old; }); it('should throw error when no promise implementation are found', function() { url = 'http://example.com/'; var old = fetch.Promise; fetch.Promise = undefined; expect(function() { fetch(url) }).to.throw(Error); fetch.Promise = old; }); it('should expose Headers, Response and Request constructors', function() { expect(fetch.Headers).to.equal(Headers); expect(fetch.Response).to.equal(Response); expect(fetch.Request).to.equal(Request); }); it('should reject with error if url is protocol relative', function() { url = '//example.com/'; return expect(fetch(url)).to.eventually.be.rejectedWith(Error); }); it('should reject with error if url is relative path', function() { url = '/some/path'; return expect(fetch(url)).to.eventually.be.rejectedWith(Error); }); it('should reject with error if protocol is unsupported', function() { url = 'ftp://example.com/'; return expect(fetch(url)).to.eventually.be.rejectedWith(Error); }); it('should reject with error on network failure', function() { url = 'http://localhost:50000/'; return expect(fetch(url)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.include({ type: 'system', code: 'ECONNREFUSED', errno: 'ECONNREFUSED' }); }); it('should resolve into response', function() { url = base + '/hello'; return fetch(url).then(function(res) { expect(res).to.be.an.instanceof(Response); expect(res.headers).to.be.an.instanceof(Headers); expect(res.body).to.be.an.instanceof(stream.Transform); expect(res.bodyUsed).to.be.false; expect(res.url).to.equal(url); expect(res.ok).to.be.true; expect(res.status).to.equal(200); expect(res.statusText).to.equal('OK'); }); }); it('should accept plain text response', function() { url = base + '/plain'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(res.bodyUsed).to.be.true; expect(result).to.be.a('string'); expect(result).to.equal('text'); }); }); }); it('should accept html response (like plain text)', function() { url = base + '/html'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/html'); return res.text().then(function(result) { expect(res.bodyUsed).to.be.true; expect(result).to.be.a('string'); expect(result).to.equal(''); }); }); }); it('should accept json response', function() { url = base + '/json'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('application/json'); return res.json().then(function(result) { expect(res.bodyUsed).to.be.true; expect(result).to.be.an('object'); expect(result).to.deep.equal({ name: 'value' }); }); }); }); it('should send request with custom headers', function() { url = base + '/inspect'; opts = { headers: { 'x-custom-header': 'abc' } }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.headers['x-custom-header']).to.equal('abc'); }); }); it('should accept headers instance', function() { url = base + '/inspect'; opts = { headers: new Headers({ 'x-custom-header': 'abc' }) }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.headers['x-custom-header']).to.equal('abc'); }); }); it('should accept custom host header', function() { url = base + '/inspect'; opts = { headers: { host: 'example.com' } }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.headers['host']).to.equal('example.com'); }); }); it('should follow redirect code 301', function() { url = base + '/redirect/301'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); expect(res.ok).to.be.true; }); }); it('should follow redirect code 302', function() { url = base + '/redirect/302'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should follow redirect code 303', function() { url = base + '/redirect/303'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should follow redirect code 307', function() { url = base + '/redirect/307'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should follow redirect code 308', function() { url = base + '/redirect/308'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should follow redirect chain', function() { url = base + '/redirect/chain'; return fetch(url).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should follow POST request redirect code 301 with GET', function() { url = base + '/redirect/301'; opts = { method: 'POST' , body: 'a=1' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); return res.json().then(function(result) { expect(result.method).to.equal('GET'); expect(result.body).to.equal(''); }); }); }); it('should follow POST request redirect code 302 with GET', function() { url = base + '/redirect/302'; opts = { method: 'POST' , body: 'a=1' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); return res.json().then(function(result) { expect(result.method).to.equal('GET'); expect(result.body).to.equal(''); }); }); }); it('should follow redirect code 303 with GET', function() { url = base + '/redirect/303'; opts = { method: 'PUT' , body: 'a=1' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); return res.json().then(function(result) { expect(result.method).to.equal('GET'); expect(result.body).to.equal(''); }); }); }); it('should obey maximum redirect, reject case', function() { url = base + '/redirect/chain'; opts = { follow: 1 } return expect(fetch(url, opts)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'max-redirect'); }); it('should obey redirect chain, resolve case', function() { url = base + '/redirect/chain'; opts = { follow: 2 } return fetch(url, opts).then(function(res) { expect(res.url).to.equal(base + '/inspect'); expect(res.status).to.equal(200); }); }); it('should allow not following redirect', function() { url = base + '/redirect/301'; opts = { follow: 0 } return expect(fetch(url, opts)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'max-redirect'); }); it('should support redirect mode, manual flag', function() { url = base + '/redirect/301'; opts = { redirect: 'manual' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(url); expect(res.status).to.equal(301); expect(res.headers.get('location')).to.equal(base + '/inspect'); }); }); it('should support redirect mode, error flag', function() { url = base + '/redirect/301'; opts = { redirect: 'error' }; return expect(fetch(url, opts)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'no-redirect'); }); it('should support redirect mode, manual flag when there is no redirect', function() { url = base + '/hello'; opts = { redirect: 'manual' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(url); expect(res.status).to.equal(200); expect(res.headers.get('location')).to.be.null; }); }); it('should follow redirect code 301 and keep existing headers', function() { url = base + '/redirect/301'; opts = { headers: new Headers({ 'x-custom-header': 'abc' }) }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(base + '/inspect'); return res.json(); }).then(function(res) { expect(res.headers['x-custom-header']).to.equal('abc'); }); }); it('should reject broken redirect', function() { url = base + '/error/redirect'; return expect(fetch(url)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'invalid-redirect'); }); it('should not reject broken redirect under manual redirect', function() { url = base + '/error/redirect'; opts = { redirect: 'manual' }; return fetch(url, opts).then(function(res) { expect(res.url).to.equal(url); expect(res.status).to.equal(301); expect(res.headers.get('location')).to.be.null; }); }); it('should handle client-error response', function() { url = base + '/error/400'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); expect(res.status).to.equal(400); expect(res.statusText).to.equal('Bad Request'); expect(res.ok).to.be.false; return res.text().then(function(result) { expect(res.bodyUsed).to.be.true; expect(result).to.be.a('string'); expect(result).to.equal('client error'); }); }); }); it('should handle server-error response', function() { url = base + '/error/500'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); expect(res.status).to.equal(500); expect(res.statusText).to.equal('Internal Server Error'); expect(res.ok).to.be.false; return res.text().then(function(result) { expect(res.bodyUsed).to.be.true; expect(result).to.be.a('string'); expect(result).to.equal('server error'); }); }); }); it('should handle network-error response', function() { url = base + '/error/reset'; return expect(fetch(url)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('code', 'ECONNRESET'); }); it('should handle DNS-error response', function() { url = 'http://domain.invalid'; return expect(fetch(url)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('code', 'ENOTFOUND'); }); it('should reject invalid json response', function() { url = base + '/error/json'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('application/json'); return expect(res.json()).to.eventually.be.rejectedWith(Error); }); }); it('should handle no content response', function() { url = base + '/no-content'; return fetch(url).then(function(res) { expect(res.status).to.equal(204); expect(res.statusText).to.equal('No Content'); expect(res.ok).to.be.true; return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.be.empty; }); }); }); it('should throw on no-content json response', function() { url = base + '/no-content'; return fetch(url).then(function(res) { return expect(res.json()).to.eventually.be.rejectedWith(FetchError); }); }); it('should handle no content response with gzip encoding', function() { url = base + '/no-content/gzip'; return fetch(url).then(function(res) { expect(res.status).to.equal(204); expect(res.statusText).to.equal('No Content'); expect(res.headers.get('content-encoding')).to.equal('gzip'); expect(res.ok).to.be.true; return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.be.empty; }); }); }); it('should handle not modified response', function() { url = base + '/not-modified'; return fetch(url).then(function(res) { expect(res.status).to.equal(304); expect(res.statusText).to.equal('Not Modified'); expect(res.ok).to.be.false; return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.be.empty; }); }); }); it('should handle not modified response with gzip encoding', function() { url = base + '/not-modified/gzip'; return fetch(url).then(function(res) { expect(res.status).to.equal(304); expect(res.statusText).to.equal('Not Modified'); expect(res.headers.get('content-encoding')).to.equal('gzip'); expect(res.ok).to.be.false; return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.be.empty; }); }); }); it('should decompress gzip response', function() { url = base + '/gzip'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.equal('hello world'); }); }); }); it('should decompress deflate response', function() { url = base + '/deflate'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.equal('hello world'); }); }); }); it('should decompress deflate raw response from old apache server', function() { url = base + '/deflate-raw'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.equal('hello world'); }); }); }); it('should skip decompression if unsupported', function() { url = base + '/sdch'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.equal('fake sdch string'); }); }); }); it('should reject if response compression is invalid', function() { url = base + '/invalid-content-encoding'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return expect(res.text()).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('code', 'Z_DATA_ERROR'); }); }); it('should allow disabling auto decompression', function() { url = base + '/gzip'; opts = { compress: false }; return fetch(url, opts).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(result).to.be.a('string'); expect(result).to.not.equal('hello world'); }); }); }); it('should allow custom timeout', function() { this.timeout(500); url = base + '/timeout'; opts = { timeout: 100 }; return expect(fetch(url, opts)).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'request-timeout'); }); it('should allow custom timeout on response body', function() { this.timeout(500); url = base + '/slow'; opts = { timeout: 100 }; return fetch(url, opts).then(function(res) { expect(res.ok).to.be.true; return expect(res.text()).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'body-timeout'); }); }); it('should clear internal timeout on fetch response', function (done) { this.timeout(1000); spawn('node', ['-e', 'require("./")("' + base + '/hello", { timeout: 5000 })']) .on('exit', function () { done(); }); }); it('should clear internal timeout on fetch redirect', function (done) { this.timeout(1000); spawn('node', ['-e', 'require("./")("' + base + '/redirect/301", { timeout: 5000 })']) .on('exit', function () { done(); }); }); it('should clear internal timeout on fetch error', function (done) { this.timeout(1000); spawn('node', ['-e', 'require("./")("' + base + '/error/reset", { timeout: 5000 })']) .on('exit', function () { done(); }); }); it('should allow POST request', function() { url = base + '/inspect'; opts = { method: 'POST' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.headers['transfer-encoding']).to.be.undefined; expect(res.headers['content-length']).to.equal('0'); }); }); it('should allow POST request with string body', function() { url = base + '/inspect'; opts = { method: 'POST' , body: 'a=1' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.body).to.equal('a=1'); expect(res.headers['transfer-encoding']).to.be.undefined; expect(res.headers['content-length']).to.equal('3'); }); }); it('should allow POST request with buffer body', function() { url = base + '/inspect'; opts = { method: 'POST' , body: new Buffer('a=1', 'utf-8') }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.body).to.equal('a=1'); expect(res.headers['transfer-encoding']).to.equal('chunked'); expect(res.headers['content-length']).to.be.undefined; }); }); it('should allow POST request with readable stream as body', function() { var body = resumer().queue('a=1').end(); body = body.pipe(new stream.PassThrough()); url = base + '/inspect'; opts = { method: 'POST' , body: body }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.body).to.equal('a=1'); expect(res.headers['transfer-encoding']).to.equal('chunked'); expect(res.headers['content-length']).to.be.undefined; }); }); it('should allow POST request with form-data as body', function() { var form = new FormData(); form.append('a','1'); url = base + '/multipart'; opts = { method: 'POST' , body: form }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.headers['content-type']).to.contain('multipart/form-data'); expect(res.headers['content-length']).to.be.a('string'); expect(res.body).to.equal('a=1'); }); }); it('should allow POST request with form-data using stream as body', function() { var form = new FormData(); form.append('my_field', fs.createReadStream('test/dummy.txt')); url = base + '/multipart'; opts = { method: 'POST' , body: form }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.headers['content-type']).to.contain('multipart/form-data'); expect(res.headers['content-length']).to.be.undefined; expect(res.body).to.contain('my_field='); }); }); it('should allow POST request with form-data as body and custom headers', function() { var form = new FormData(); form.append('a','1'); var headers = form.getHeaders(); headers['b'] = '2'; url = base + '/multipart'; opts = { method: 'POST' , body: form , headers: headers }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.headers['content-type']).to.contain('multipart/form-data'); expect(res.headers['content-length']).to.be.a('string'); expect(res.headers.b).to.equal('2'); expect(res.body).to.equal('a=1'); }); }); it('should allow POST request with object body', function() { url = base + '/inspect'; // note that fetch simply calls tostring on an object opts = { method: 'POST' , body: { a:1 } }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.body).to.equal('[object Object]'); }); }); it('should allow PUT request', function() { url = base + '/inspect'; opts = { method: 'PUT' , body: 'a=1' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('PUT'); expect(res.body).to.equal('a=1'); }); }); it('should allow DELETE request', function() { url = base + '/inspect'; opts = { method: 'DELETE' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('DELETE'); }); }); it('should allow POST request with string body', function() { url = base + '/inspect'; opts = { method: 'POST' , body: 'a=1' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('POST'); expect(res.body).to.equal('a=1'); expect(res.headers['transfer-encoding']).to.be.undefined; expect(res.headers['content-length']).to.equal('3'); }); }); it('should allow DELETE request with string body', function() { url = base + '/inspect'; opts = { method: 'DELETE' , body: 'a=1' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('DELETE'); expect(res.body).to.equal('a=1'); expect(res.headers['transfer-encoding']).to.be.undefined; expect(res.headers['content-length']).to.equal('3'); }); }); it('should allow PATCH request', function() { url = base + '/inspect'; opts = { method: 'PATCH' , body: 'a=1' }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.method).to.equal('PATCH'); expect(res.body).to.equal('a=1'); }); }); it('should allow HEAD request', function() { url = base + '/hello'; opts = { method: 'HEAD' }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(200); expect(res.statusText).to.equal('OK'); expect(res.headers.get('content-type')).to.equal('text/plain'); expect(res.body).to.be.an.instanceof(stream.Transform); return res.text(); }).then(function(text) { expect(text).to.equal(''); }); }); it('should allow HEAD request with content-encoding header', function() { url = base + '/error/404'; opts = { method: 'HEAD' }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(404); expect(res.headers.get('content-encoding')).to.equal('gzip'); return res.text(); }).then(function(text) { expect(text).to.equal(''); }); }); it('should allow OPTIONS request', function() { url = base + '/options'; opts = { method: 'OPTIONS' }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(200); expect(res.statusText).to.equal('OK'); expect(res.headers.get('allow')).to.equal('GET, HEAD, OPTIONS'); expect(res.body).to.be.an.instanceof(stream.Transform); }); }); it('should reject decoding body twice', function() { url = base + '/plain'; return fetch(url).then(function(res) { expect(res.headers.get('content-type')).to.equal('text/plain'); return res.text().then(function(result) { expect(res.bodyUsed).to.be.true; return expect(res.text()).to.eventually.be.rejectedWith(Error); }); }); }); it('should support maximum response size, multiple chunk', function() { url = base + '/size/chunk'; opts = { size: 5 }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(200); expect(res.headers.get('content-type')).to.equal('text/plain'); return expect(res.text()).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'max-size'); }); }); it('should support maximum response size, single chunk', function() { url = base + '/size/long'; opts = { size: 5 }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(200); expect(res.headers.get('content-type')).to.equal('text/plain'); return expect(res.text()).to.eventually.be.rejected .and.be.an.instanceOf(FetchError) .and.have.property('type', 'max-size'); }); }); it('should support encoding decode, xml dtd detect', function() { url = base + '/encoding/euc-jp'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('日本語'); }); }); }); it('should support encoding decode, content-type detect', function() { url = base + '/encoding/shift-jis'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('
日本語
'); }); }); }); it('should support encoding decode, html5 detect', function() { url = base + '/encoding/gbk'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('
中文
'); }); }); }); it('should support encoding decode, html4 detect', function() { url = base + '/encoding/gb2312'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('
中文
'); }); }); }); it('should default to utf8 encoding', function() { url = base + '/encoding/utf8'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); expect(res.headers.get('content-type')).to.be.null; return res.text().then(function(result) { expect(result).to.equal('中文'); }); }); }); it('should support uncommon content-type order, charset in front', function() { url = base + '/encoding/order1'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('中文'); }); }); }); it('should support uncommon content-type order, end with qs', function() { url = base + '/encoding/order2'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); return res.text().then(function(result) { expect(result).to.equal('中文'); }); }); }); it('should support chunked encoding, html4 detect', function() { url = base + '/encoding/chunked'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); // because node v0.12 doesn't have str.repeat var padding = new Array(10 + 1).join('a'); return res.text().then(function(result) { expect(result).to.equal(padding + '
日本語
'); }); }); }); it('should only do encoding detection up to 1024 bytes', function() { url = base + '/encoding/invalid'; return fetch(url).then(function(res) { expect(res.status).to.equal(200); // because node v0.12 doesn't have str.repeat var padding = new Array(1200 + 1).join('a'); return res.text().then(function(result) { expect(result).to.not.equal(padding + '中文'); }); }); }); it('should allow piping response body as stream', function(done) { url = base + '/hello'; fetch(url).then(function(res) { expect(res.body).to.be.an.instanceof(stream.Transform); res.body.on('data', function(chunk) { if (chunk === null) { return; } expect(chunk.toString()).to.equal('world'); }); res.body.on('end', function() { done(); }); }); }); it('should allow cloning a response, and use both as stream', function(done) { url = base + '/hello'; return fetch(url).then(function(res) { var counter = 0; var r1 = res.clone(); expect(res.body).to.be.an.instanceof(stream.Transform); expect(r1.body).to.be.an.instanceof(stream.Transform); res.body.on('data', function(chunk) { if (chunk === null) { return; } expect(chunk.toString()).to.equal('world'); }); res.body.on('end', function() { counter++; if (counter == 2) { done(); } }); r1.body.on('data', function(chunk) { if (chunk === null) { return; } expect(chunk.toString()).to.equal('world'); }); r1.body.on('end', function() { counter++; if (counter == 2) { done(); } }); }); }); it('should allow cloning a json response and log it as text response', function() { url = base + '/json'; return fetch(url).then(function(res) { var r1 = res.clone(); return fetch.Promise.all([res.json(), r1.text()]).then(function(results) { expect(results[0]).to.deep.equal({name: 'value'}); expect(results[1]).to.equal('{"name":"value"}'); }); }); }); it('should allow cloning a json response, and then log it as text response', function() { url = base + '/json'; return fetch(url).then(function(res) { var r1 = res.clone(); return res.json().then(function(result) { expect(result).to.deep.equal({name: 'value'}); return r1.text().then(function(result) { expect(result).to.equal('{"name":"value"}'); }); }); }); }); it('should allow cloning a json response, first log as text response, then return json object', function() { url = base + '/json'; return fetch(url).then(function(res) { var r1 = res.clone(); return r1.text().then(function(result) { expect(result).to.equal('{"name":"value"}'); return res.json().then(function(result) { expect(result).to.deep.equal({name: 'value'}); }); }); }); }); it('should not allow cloning a response after its been used', function() { url = base + '/hello'; return fetch(url).then(function(res) { return res.text().then(function(result) { expect(function() { var r1 = res.clone(); }).to.throw(Error); }); }) }); it('should allow get all responses of a header', function() { url = base + '/cookie'; return fetch(url).then(function(res) { expect(res.headers.get('set-cookie')).to.equal('a=1'); expect(res.headers.get('Set-Cookie')).to.equal('a=1'); expect(res.headers.getAll('set-cookie')).to.deep.equal(['a=1', 'b=1']); expect(res.headers.getAll('Set-Cookie')).to.deep.equal(['a=1', 'b=1']); }); }); it('should allow iterating through all headers', function() { var headers = new Headers({ a: 1 , b: [2, 3] , c: [4] }); expect(headers).to.have.property('forEach'); var result = []; headers.forEach(function(val, key) { result.push([key, val]); }); expected = [ ["a", "1"] , ["b", "2"] , ["b", "3"] , ["c", "4"] ]; expect(result).to.deep.equal(expected); }); it('should allow deleting header', function() { url = base + '/cookie'; return fetch(url).then(function(res) { res.headers.delete('set-cookie'); expect(res.headers.get('set-cookie')).to.be.null; expect(res.headers.getAll('set-cookie')).to.be.empty; }); }); it('should send request with connection keep-alive if agent is provided', function() { url = base + '/inspect'; opts = { agent: new http.Agent({ keepAlive: true }) }; return fetch(url, opts).then(function(res) { return res.json(); }).then(function(res) { expect(res.headers['connection']).to.equal('keep-alive'); }); }); it('should ignore unsupported attributes while reading headers', function() { var FakeHeader = function() {}; // prototypes are ignored FakeHeader.prototype.z = 'fake'; var res = new FakeHeader; // valid res.a = 'string'; res.b = ['1','2']; res.c = ''; res.d = []; // common mistakes, normalized res.e = 1; res.f = [1, 2]; // invalid, ignored res.g = { a:1 }; res.h = undefined; res.i = null; res.j = NaN; res.k = true; res.l = false; res.m = new Buffer('test'); var h1 = new Headers(res); expect(h1._headers['a']).to.include('string'); expect(h1._headers['b']).to.include('1'); expect(h1._headers['b']).to.include('2'); expect(h1._headers['c']).to.include(''); expect(h1._headers['d']).to.be.undefined; expect(h1._headers['e']).to.include('1'); expect(h1._headers['f']).to.include('1'); expect(h1._headers['f']).to.include('2'); expect(h1._headers['g']).to.be.undefined; expect(h1._headers['h']).to.be.undefined; expect(h1._headers['i']).to.be.undefined; expect(h1._headers['j']).to.be.undefined; expect(h1._headers['k']).to.be.undefined; expect(h1._headers['l']).to.be.undefined; expect(h1._headers['m']).to.be.undefined; expect(h1._headers['z']).to.be.undefined; }); it('should wrap headers', function() { var h1 = new Headers({ a: '1' }); var h2 = new Headers(h1); h2.set('b', '1'); var h3 = new Headers(h2); h3.append('a', '2'); expect(h1._headers['a']).to.include('1'); expect(h1._headers['a']).to.not.include('2'); expect(h2._headers['a']).to.include('1'); expect(h2._headers['a']).to.not.include('2'); expect(h2._headers['b']).to.include('1'); expect(h3._headers['a']).to.include('1'); expect(h3._headers['a']).to.include('2'); expect(h3._headers['b']).to.include('1'); }); it('should support fetch with Request instance', function() { url = base + '/hello'; var req = new Request(url); return fetch(req).then(function(res) { expect(res.url).to.equal(url); expect(res.ok).to.be.true; expect(res.status).to.equal(200); }); }); it('should support wrapping Request instance', function() { url = base + '/hello'; var form = new FormData(); form.append('a', '1'); var r1 = new Request(url, { method: 'POST' , follow: 1 , body: form }); var r2 = new Request(r1, { follow: 2 }); expect(r2.url).to.equal(url); expect(r2.method).to.equal('POST'); // note that we didn't clone the body expect(r2.body).to.equal(form); expect(r1.follow).to.equal(1); expect(r2.follow).to.equal(2); expect(r1.counter).to.equal(0); expect(r2.counter).to.equal(0); }); it('should support overwrite Request instance', function() { url = base + '/inspect'; var req = new Request(url, { method: 'POST' , headers: { a: '1' } }); return fetch(req, { method: 'GET' , headers: { a: '2' } }).then(function(res) { return res.json(); }).then(function(body) { expect(body.method).to.equal('GET'); expect(body.headers.a).to.equal('2'); }); }); it('should support empty options in Response constructor', function() { var body = resumer().queue('a=1').end(); body = body.pipe(new stream.PassThrough()); var res = new Response(body); return res.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support parsing headers in Response constructor', function() { var res = new Response(null, { headers: { a: '1' } }); expect(res.headers.get('a')).to.equal('1'); }); it('should support text() method in Response constructor', function() { var res = new Response('a=1'); return res.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support json() method in Response constructor', function() { var res = new Response('{"a":1}'); return res.json().then(function(result) { expect(result.a).to.equal(1); }); }); it('should support buffer() method in Response constructor', function() { var res = new Response('a=1'); return res.buffer().then(function(result) { expect(result.toString()).to.equal('a=1'); }); }); it('should support clone() method in Response constructor', function() { var body = resumer().queue('a=1').end(); body = body.pipe(new stream.PassThrough()); var res = new Response(body, { headers: { a: '1' } , url: base , status: 346 , statusText: 'production' }); var cl = res.clone(); expect(cl.headers.get('a')).to.equal('1'); expect(cl.url).to.equal(base); expect(cl.status).to.equal(346); expect(cl.statusText).to.equal('production'); expect(cl.ok).to.be.false; // clone body shouldn't be the same body expect(cl.body).to.not.equal(body); return cl.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support stream as body in Response constructor', function() { var body = resumer().queue('a=1').end(); body = body.pipe(new stream.PassThrough()); var res = new Response(body); return res.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support string as body in Response constructor', function() { var res = new Response('a=1'); return res.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support buffer as body in Response constructor', function() { var res = new Response(new Buffer('a=1')); return res.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should default to 200 as status code', function() { var res = new Response(null); expect(res.status).to.equal(200); }); it('should support parsing headers in Request constructor', function() { url = base; var req = new Request(url, { headers: { a: '1' } }); expect(req.url).to.equal(url); expect(req.headers.get('a')).to.equal('1'); }); it('should support text() method in Request constructor', function() { url = base; var req = new Request(url, { body: 'a=1' }); expect(req.url).to.equal(url); return req.text().then(function(result) { expect(result).to.equal('a=1'); }); }); it('should support json() method in Request constructor', function() { url = base; var req = new Request(url, { body: '{"a":1}' }); expect(req.url).to.equal(url); return req.json().then(function(result) { expect(result.a).to.equal(1); }); }); it('should support buffer() method in Request constructor', function() { url = base; var req = new Request(url, { body: 'a=1' }); expect(req.url).to.equal(url); return req.buffer().then(function(result) { expect(result.toString()).to.equal('a=1'); }); }); it('should support arbitrary url in Request constructor', function() { url = 'anything'; var req = new Request(url); expect(req.url).to.equal('anything'); }); it('should support clone() method in Request constructor', function() { url = base; var body = resumer().queue('a=1').end(); body = body.pipe(new stream.PassThrough()); var agent = new http.Agent(); var req = new Request(url, { body: body , method: 'POST' , redirect: 'manual' , headers: { b: '2' } , follow: 3 , compress: false , agent: agent }); var cl = req.clone(); expect(cl.url).to.equal(url); expect(cl.method).to.equal('POST'); expect(cl.redirect).to.equal('manual'); expect(cl.headers.get('b')).to.equal('2'); expect(cl.follow).to.equal(3); expect(cl.compress).to.equal(false); expect(cl.method).to.equal('POST'); expect(cl.counter).to.equal(0); expect(cl.agent).to.equal(agent); // clone body shouldn't be the same body expect(cl.body).to.not.equal(body); return fetch.Promise.all([cl.text(), req.text()]).then(function(results) { expect(results[0]).to.equal('a=1'); expect(results[1]).to.equal('a=1'); }); }); it('should support text(), json() and buffer() method in Body constructor', function() { var body = new Body('a=1'); expect(body).to.have.property('text'); expect(body).to.have.property('json'); expect(body).to.have.property('buffer'); }); it('should create custom FetchError', function funcName() { var systemError = new Error('system'); systemError.code = 'ESOMEERROR'; var err = new FetchError('test message', 'test-error', systemError); expect(err).to.be.an.instanceof(Error); expect(err).to.be.an.instanceof(FetchError); expect(err.name).to.equal('FetchError'); expect(err.message).to.equal('test message'); expect(err.type).to.equal('test-error'); expect(err.code).to.equal('ESOMEERROR'); expect(err.errno).to.equal('ESOMEERROR'); expect(err.stack).to.include('funcName'); expect(err.stack.split('\n')[0]).to.equal(err.name + ': ' + err.message); }); it('should support https request', function() { this.timeout(5000); url = 'https://github.com/'; opts = { method: 'HEAD' }; return fetch(url, opts).then(function(res) { expect(res.status).to.equal(200); expect(res.ok).to.be.true; }); }); });