2.8.2

Instead of right when response is created, which misses timeouts
that occur later

.airtap.yml

@@ -0,0 +1,23 @@
+sauce_connect: true
+browsers:
+  - name: chrome
+    version: 39..latest
+  - name: firefox
+    version: 34..latest
+  - name: safari
+    version: 8..latest
+  - name: MicrosoftEdge
+    version: 13..latest
+  - name: ie
+    version: 9..latest
+  - name: iphone
+    version: '9.3..latest'
+  - name: android
+    version: '4.4..6.0' # TODO: change this back to latest once https://github.com/airtap/browsers/issues/3 is fixed
+server: ./test/server/index.js
+scripts:
+  - "/ie8-polyfill.js"
+  - "/test-polyfill.js"
+browserify:
+  - options:
+      dedupe: false

.gitignore

@@ -2,4 +2,4 @@
 bundle.js
 node_modules
 npm-debug.log
-.zuulrc
+.airtaprc

.travis.yml

@@ -1,3 +1,7 @@
 language: node_js
 node_js:
-  - "4.1"
+  - "node"
+addons:
+  sauce_connect: true
+  hosts:
+    - airtap.local

.zuul.yml

@@ -1,22 +0,0 @@
-ui: tape
-browsers:
-  - name: chrome
-    version: 39..latest
-  - name: firefox
-    version: 34..latest
-  - name: safari
-    version: 5..latest
-  - name: microsoftedge
-    version: 20..latest
-  - name: ie
-    version: 9..latest
-  - name: opera
-    version: 11..latest
-  - name: iphone
-    version: 5.1..latest
-  - name: android
-    version: 4.0..latest
-server: ./test/server/index.js
-scripts:
-  - "/ie8-polyfill.js"
-  - "/test-polyfill.js"

README.md

@@ -2,20 +2,24 @@
 
 [![Sauce Test Status](https://saucelabs.com/browser-matrix/stream-http.svg)](https://saucelabs.com/u/stream-http)
 
-This module is an implementation of node's native `http` module for the browser.
-It tries to match node's api and behavior as closely as possible, but some features
+This module is an implementation of Node's native `http` module for the browser.
+It tries to match Node's API and behavior as closely as possible, but some features
 aren't available, since browsers don't give nearly as much control over requests.
 
-This is heavily inspired by, and intended to replace, [http-browserify](https://github.com/substack/http-browserify)
+This is heavily inspired by, and intended to replace, [http-browserify](https://github.com/substack/http-browserify).
 
 ## What does it do?
 
 In accordance with its name, `stream-http` tries to provide data to its caller before
 the request has completed whenever possible.
 
+Backpressure, allowing the browser to only pull data from the server as fast as it is
+consumed, is supported in:
+* Chrome >= 58 (using `fetch` and `WritableStream`)
+
 The following browsers support true streaming, where only a small amount of the request
 has to be held in memory at once:
-* Chrome >= 43 (using the `fetch` api)
+* Chrome >= 43 (using the `fetch` API)
 * Firefox >= 9 (using `moz-chunked-arraybuffer` responseType with xhr)
 
 The following browsers support pseudo-streaming, where the data is available before the
@@ -38,43 +42,55 @@ All browsers with full ES5 support shouldn't require any polyfills.
 
 ## How do you use it?
 
-The intent is to have the same api as the client part of the
-[node HTTP module](https://nodejs.org/api/http.html). The interfaces are the same wherever
-practical, although limitations in browsers make an exact clone of the node api impossible.
+The intent is to have the same API as the client part of the
+[Node HTTP module](https://nodejs.org/api/http.html). The interfaces are the same wherever
+practical, although limitations in browsers make an exact clone of the Node API impossible.
 
 This module implements `http.request`, `http.get`, and most of `http.ClientRequest`
 and `http.IncomingMessage` in addition to `http.METHODS` and `http.STATUS_CODES`. See the
-node docs for how these work.
+Node docs for how these work.
 
-### Extra features compared to node
+### Extra features compared to Node
+
+* The `message.url` property provides access to the final URL after all redirects. This
+is useful since the browser follows all redirects silently, unlike Node. It is available
+in Chrome 37 and newer, Firefox 32 and newer, and Safari 9 and newer.
 
 * The `options.withCredentials` boolean flag, used to indicate if the browser should send
 cookies or authentication information with a CORS request. Default false.
 
 This module has to make some tradeoffs to support binary data and/or streaming. Generally,
 the module can make a fairly good decision about which underlying browser features to use,
-but sometimes it helps to get a little input from the user.
+but sometimes it helps to get a little input from the developer.
 
 * The `options.mode` field passed into `http.request` or `http.get` can take on one of the
 following values:
-  * 'default' (or any falsy value, including undefined): Try to provide partial data before
+  * 'default' (or any falsy value, including `undefined`): Try to provide partial data before
 the request completes, but not at the cost of correctness for binary data or correctness of
 the 'content-type' response header. This mode will also avoid slower code paths whenever
 possible, which is particularly useful when making large requests in a browser like Safari
-that has a weaker javascript engine.
+that has a weaker JavaScript engine.
   * 'allow-wrong-content-type': Provides partial data in more cases than 'default', but
 at the expense of causing the 'content-type' response header to be incorrectly reported
 (as 'text/plain; charset=x-user-defined') in some browsers, notably Safari and Chrome 42
 and older. Preserves binary data whenever possible. In some cases the implementation may
 also be a bit slow. This was the default in versions of this module before 1.5.
   * 'prefer-stream': Provide data before the request completes even if binary data (anything
-that isn't a single-byte ASCII or utf8 character) will be corrupted. Of course, this option
+that isn't a single-byte ASCII or UTF8 character) will be corrupted. Of course, this option
 is only safe for text data. May also cause the 'content-type' response header to be
 incorrectly reported (as 'text/plain; charset=x-user-defined').
+  * 'disable-fetch': Force the use of plain XHR regardless of the browser declaring a fetch
+capability. Preserves the correctness of binary data and the 'content-type' response header.
   * 'prefer-fast': Deprecated; now a synonym for 'default', which has the same performance
 characteristics as this mode did in versions before 1.5.
 
-### Features missing compared to node
+* `options.requestTimeout` allows setting a timeout in millisecionds for XHR and fetch (if
+supported by the browser). This is a limit on how long the entire process takes from
+beginning to end. Note that this is not the same as the node `setTimeout` functions,
+which apply to pauses in data transfer over the underlying socket, or the node `timeout`
+option, which applies to opening the connection.
+
+### Features missing compared to Node
 
 * `http.Agent` is only a stub
 * The 'socket', 'connect', 'upgrade', and 'continue' events on `http.ClientRequest`.
@@ -86,6 +102,10 @@ certain headers.
 * `message.rawHeaders` is modified by the browser, and may not quite match what is sent by
 the server.
 * `message.trailers` and `message.rawTrailers` will remain empty.
+* Redirects are followed silently by the browser, so it isn't possible to access the 301/302
+redirect pages.
+* The `timeout` event/option and `setTimeout` functions, which operate on the underlying
+socket, are not available. However, see `options.requestTimeout` above.
 
 ## Example
 
@@ -93,11 +113,11 @@ the server.
 http.get('/bundle.js', function (res) {
 	var div = document.getElementById('result');
 	div.innerHTML += 'GET /beep<br>';
-	
+
 	res.on('data', function (buf) {
 		div.innerHTML += buf;
 	});
-	
+
 	res.on('end', function () {
 		div.innerHTML += '<br>__END__';
 	});
@@ -106,7 +126,7 @@ http.get('/bundle.js', function (res) {
 
 ## Running tests
 
-There are two sets of tests: the tests that run in node (found in `test/node`) and the tests
+There are two sets of tests: the tests that run in Node (found in `test/node`) and the tests
 that run in the browser (found in `test/browser`). Normally the browser tests run on
 [Sauce Labs](http://saucelabs.com/).
 
@@ -114,7 +134,7 @@ Running `npm test` will run both sets of tests, but in order for the Sauce Labs
 you will need to sign up for an account (free for open source projects) and put the
 credentials in a [`.zuulrc` file](https://github.com/defunctzombie/zuul/wiki/zuulrc).
 
-To run just the node tests, run `npm run test-node`.
+To run just the Node tests, run `npm run test-node`.
 
 To run the browser tests locally, run `npm run test-browser-local` and point your browser to
 `http://localhost:8080/__zuul`

index.js

@@ -1,4 +1,5 @@
 var ClientRequest = require('./lib/request')
+var response = require('./lib/response')
 var extend = require('xtend')
 var statusCodes = require('builtin-status-codes')
 var url = require('url')
@@ -44,9 +45,14 @@ http.get = function get (opts, cb) {
 	return req
 }
 
+http.ClientRequest = ClientRequest
+http.IncomingMessage = response.IncomingMessage
+
 http.Agent = function () {}
 http.Agent.defaultMaxSockets = 4
 
+http.globalAgent = new http.Agent()
+
 http.STATUS_CODES = statusCodes
 
 http.METHODS = [

lib/capability.js

@@ -1,4 +1,8 @@
-exports.fetch = isFunction(global.fetch) && isFunction(global.ReadableByteStream)
+exports.fetch = isFunction(global.fetch) && isFunction(global.ReadableStream)
+
+exports.writableStream = isFunction(global.WritableStream)
+
+exports.abortController = isFunction(global.AbortController)
 
 exports.blobConstructor = false
 try {
@@ -6,12 +10,34 @@ try {
 	exports.blobConstructor = true
 } catch (e) {}
 
-var xhr = new global.XMLHttpRequest()
-// If location.host is empty, e.g. if this page/worker was loaded
-// from a Blob, then use example.com to avoid an error
-xhr.open('GET', global.location.host ? '/' : 'https://example.com')
+// The xhr request to example.com may violate some restrictive CSP configurations,
+// so if we're running in a browser that supports `fetch`, avoid calling getXHR()
+// and assume support for certain features below.
+var xhr
+function getXHR () {
+	// Cache the xhr value
+	if (xhr !== undefined) return xhr
+
+	if (global.XMLHttpRequest) {
+		xhr = new global.XMLHttpRequest()
+		// If XDomainRequest is available (ie only, where xhr might not work
+		// cross domain), use the page location. Otherwise use example.com
+		// Note: this doesn't actually make an http request.
+		try {
+			xhr.open('GET', global.XDomainRequest ? '/' : 'https://example.com')
+		} catch(e) {
+			xhr = null
+		}
+	} else {
+		// Service workers don't have XHR
+		xhr = null
+	}
+	return xhr
+}
 
 function checkTypeSupport (type) {
+	var xhr = getXHR()
+	if (!xhr) return false
 	try {
 		xhr.responseType = type
 		return xhr.responseType === type
@@ -24,17 +50,24 @@ function checkTypeSupport (type) {
 var haveArrayBuffer = typeof global.ArrayBuffer !== 'undefined'
 var haveSlice = haveArrayBuffer && isFunction(global.ArrayBuffer.prototype.slice)
 
-exports.arraybuffer = haveArrayBuffer && checkTypeSupport('arraybuffer')
+// If fetch is supported, then arraybuffer will be supported too. Skip calling
+// checkTypeSupport(), since that calls getXHR().
+exports.arraybuffer = exports.fetch || (haveArrayBuffer && checkTypeSupport('arraybuffer'))
+
 // These next two tests unavoidably show warnings in Chrome. Since fetch will always
 // be used if it's available, just return false for these to avoid the warnings.
 exports.msstream = !exports.fetch && haveSlice && checkTypeSupport('ms-stream')
 exports.mozchunkedarraybuffer = !exports.fetch && haveArrayBuffer &&
 	checkTypeSupport('moz-chunked-arraybuffer')
-exports.overrideMimeType = isFunction(xhr.overrideMimeType)
+
+// If fetch is supported, then overrideMimeType will be supported too. Skip calling
+// getXHR().
+exports.overrideMimeType = exports.fetch || (getXHR() ? isFunction(getXHR().overrideMimeType) : false)
+
 exports.vbArray = isFunction(global.VBArray)
 
 function isFunction (value) {
-  return typeof value === 'function'
+	return typeof value === 'function'
 }
 
 xhr = null // Help gc

lib/request.js

@@ -1,15 +1,14 @@
-// var Base64 = require('Base64')
 var capability = require('./capability')
 var inherits = require('inherits')
 var response = require('./response')
-var stream = require('stream')
+var stream = require('readable-stream')
 var toArrayBuffer = require('to-arraybuffer')
 
 var IncomingMessage = response.IncomingMessage
 var rStates = response.readyStates
 
-function decideMode (preferBinary) {
-	if (capability.fetch) {
+function decideMode (preferBinary, useFetch) {
+	if (capability.fetch && useFetch) {
 		return 'fetch'
 	} else if (capability.mozchunkedarraybuffer) {
 		return 'moz-chunked-arraybuffer'
@@ -38,7 +37,12 @@ var ClientRequest = module.exports = function (opts) {
 	})
 
 	var preferBinary
-	if (opts.mode === 'prefer-streaming') {
+	var useFetch = true
+	if (opts.mode === 'disable-fetch' || ('requestTimeout' in opts && !capability.abortController)) {
+		// If the use of XHR should be preferred. Not typically needed.
+		useFetch = false
+		preferBinary = true
+	} else if (opts.mode === 'prefer-streaming') {
 		// If streaming is a high priority but binary compatibility and
 		// the accuracy of the 'content-type' header aren't
 		preferBinary = false
@@ -51,7 +55,8 @@ var ClientRequest = module.exports = function (opts) {
 	} else {
 		throw new Error('Invalid value for opts.mode')
 	}
-	self._mode = decideMode(preferBinary)
+	self._mode = decideMode(preferBinary, useFetch)
+	self._fetchTimer = null
 
 	self.on('finish', function () {
 		self._onFinish()
@@ -76,8 +81,10 @@ ClientRequest.prototype.setHeader = function (name, value) {
 }
 
 ClientRequest.prototype.getHeader = function (name) {
-	var self = this
-	return self._headers[name.toLowerCase()].value
+	var header = this._headers[name.toLowerCase()]
+	if (header)
+		return header.value
+	return null
 }
 
 ClientRequest.prototype.removeHeader = function (name) {
@@ -93,9 +100,11 @@ ClientRequest.prototype._onFinish = function () {
 	var opts = self._opts
 
 	var headersObj = self._headers
-	var body
-	if (opts.method === 'POST' || opts.method === 'PUT' || opts.method === 'PATCH') {
-		if (capability.blobConstructor) {
+	var body = null
+	if (opts.method !== 'GET' && opts.method !== 'HEAD') {
+		if (capability.arraybuffer) {
+			body = toArrayBuffer(Buffer.concat(self._body))
+		} else if (capability.blobConstructor) {
 			body = new global.Blob(self._body.map(function (buffer) {
 				return toArrayBuffer(buffer)
 			}), {
@@ -107,22 +116,51 @@ ClientRequest.prototype._onFinish = function () {
 		}
 	}
 
+	// create flattened list of headers
+	var headersList = []
+	Object.keys(headersObj).forEach(function (keyName) {
+		var name = headersObj[keyName].name
+		var value = headersObj[keyName].value
+		if (Array.isArray(value)) {
+			value.forEach(function (v) {
+				headersList.push([name, v])
+			})
+		} else {
+			headersList.push([name, value])
+		}
+	})
+
 	if (self._mode === 'fetch') {
-		var headers = Object.keys(headersObj).map(function (name) {
-			return [headersObj[name].name, headersObj[name].value]
-		})
+		var signal = null
+		var fetchTimer = null
+		if (capability.abortController) {
+			var controller = new AbortController()
+			signal = controller.signal
+			self._fetchAbortController = controller
+
+			if ('requestTimeout' in opts && opts.requestTimeout !== 0) {
+				self._fetchTimer = global.setTimeout(function () {
+					self.emit('requestTimeout')
+					if (self._fetchAbortController)
+						self._fetchAbortController.abort()
+				}, opts.requestTimeout)
+			}
+		}
 
 		global.fetch(self._opts.url, {
 			method: self._opts.method,
-			headers: headers,
-			body: body,
+			headers: headersList,
+			body: body || undefined,
 			mode: 'cors',
-			credentials: opts.withCredentials ? 'include' : 'same-origin'
+			credentials: opts.withCredentials ? 'include' : 'same-origin',
+			signal: signal
 		}).then(function (response) {
 			self._fetchResponse = response
 			self._connect()
 		}, function (reason) {
-			self.emit('error', reason)
+			global.clearTimeout(self._fetchTimer)
+			if (!self._destroyed)
+				self.emit('error', reason)
 		})
 	} else {
 		var xhr = self._xhr = new global.XMLHttpRequest()
@@ -145,8 +183,15 @@ ClientRequest.prototype._onFinish = function () {
 		if (self._mode === 'text' && 'overrideMimeType' in xhr)
 			xhr.overrideMimeType('text/plain; charset=x-user-defined')
 
-		Object.keys(headersObj).forEach(function (name) {
-			xhr.setRequestHeader(headersObj[name].name, headersObj[name].value)
+		if ('requestTimeout' in opts) {
+			xhr.timeout = opts.requestTimeout
+			xhr.ontimeout = function () {
+				self.emit('requestTimeout')
+			}
+		}
+
+		headersList.forEach(function (header) {
+			xhr.setRequestHeader(header[0], header[1])
 		})
 
 		self._response = null
@@ -215,7 +260,11 @@ ClientRequest.prototype._connect = function () {
 	if (self._destroyed)
 		return
 
-	self._response = new IncomingMessage(self._xhr, self._fetchResponse, self._mode)
+	self._response = new IncomingMessage(self._xhr, self._fetchResponse, self._mode, self._fetchTimer)
+	self._response.on('error', function(err) {
+		self.emit('error', err)
+	})
+
 	self.emit('response', self._response)
 }
 
@@ -229,12 +278,13 @@ ClientRequest.prototype._write = function (chunk, encoding, cb) {
 ClientRequest.prototype.abort = ClientRequest.prototype.destroy = function () {
 	var self = this
 	self._destroyed = true
+	global.clearTimeout(self._fetchTimer)
 	if (self._response)
 		self._response._destroyed = true
 	if (self._xhr)
 		self._xhr.abort()
-	// Currently, there isn't a way to truly abort a fetch.
-	// If you like bikeshedding, see https://github.com/whatwg/fetch/issues/27
+	else if (self._fetchAbortController)
+		self._fetchAbortController.abort()
 }
 
 ClientRequest.prototype.end = function (data, encoding, cb) {

lib/response.js

@@ -1,6 +1,6 @@
 var capability = require('./capability')
 var inherits = require('inherits')
-var stream = require('stream')
+var stream = require('readable-stream')
 
 var rStates = exports.readyStates = {
 	UNSENT: 0,
@@ -10,7 +10,7 @@ var rStates = exports.readyStates = {
 	DONE: 4
 }
 
-var IncomingMessage = exports.IncomingMessage = function (xhr, response, mode) {
+var IncomingMessage = exports.IncomingMessage = function (xhr, response, mode, fetchTimer) {
 	var self = this
 	stream.Readable.call(self)
 
@@ -31,35 +31,73 @@ var IncomingMessage = exports.IncomingMessage = function (xhr, response, mode) {
 	if (mode === 'fetch') {
 		self._fetchResponse = response
 
+		self.url = response.url
 		self.statusCode = response.status
 		self.statusMessage = response.statusText
-		// backwards compatible version of for (<item> of <iterable>):
-		// for (var <item>,_i,_it = <iterable>[Symbol.iterator](); <item> = (_i = _it.next()).value,!_i.done;)
-		for (var header, _i, _it = response.headers[Symbol.iterator](); header = (_i = _it.next()).value, !_i.done;) {
-			self.headers[header[0].toLowerCase()] = header[1]
-			self.rawHeaders.push(header[0], header[1])
-		}
+		
+		response.headers.forEach(function (header, key){
+			self.headers[key.toLowerCase()] = header
+			self.rawHeaders.push(key, header)
+		})
 
-		// TODO: this doesn't respect backpressure. Once WritableStream is available, this can be fixed
+		if (capability.writableStream) {
+			var writable = new WritableStream({
+				write: function (chunk) {
+					return new Promise(function (resolve, reject) {
+						if (self._destroyed) {
+							reject()
+						} else if(self.push(new Buffer(chunk))) {
+							resolve()
+						} else {
+							self._resumeFetch = resolve
+						}
+					})
+				},
+				close: function () {
+					global.clearTimeout(fetchTimer)
+					if (!self._destroyed)
+						self.push(null)
+				},
+				abort: function (err) {
+					if (!self._destroyed)
+						self.emit('error', err)
+				}
+			})
+
+			try {
+				response.body.pipeTo(writable).catch(function (err) {
+					global.clearTimeout(fetchTimer)
+					if (!self._destroyed)
+						self.emit('error', err)
+				})
+				return
+			} catch (e) {} // pipeTo method isn't defined. Can't find a better way to feature test this
+		}
+		// fallback for when writableStream or pipeTo aren't available
 		var reader = response.body.getReader()
 		function read () {
 			reader.read().then(function (result) {
 				if (self._destroyed)
 					return
 				if (result.done) {
+					global.clearTimeout(fetchTimer)
 					self.push(null)
 					return
 				}
 				self.push(new Buffer(result.value))
 				read()
+			}).catch(function (err) {
+				global.clearTimeout(fetchTimer)
+				if (!self._destroyed)
+					self.emit('error', err)
 			})
 		}
 		read()
-
 	} else {
 		self._xhr = xhr
 		self._pos = 0
 
+		self.url = xhr.responseURL
 		self.statusCode = xhr.status
 		self.statusMessage = xhr.statusText
 		var headers = xhr.getAllResponseHeaders().split(/\r?\n/)
@@ -67,10 +105,16 @@ var IncomingMessage = exports.IncomingMessage = function (xhr, response, mode) {
 			var matches = header.match(/^([^:]+):\s*(.*)/)
 			if (matches) {
 				var key = matches[1].toLowerCase()
-				if (self.headers[key] !== undefined)
+				if (key === 'set-cookie') {
+					if (self.headers[key] === undefined) {
+						self.headers[key] = []
+					}
+					self.headers[key].push(matches[2])
+				} else if (self.headers[key] !== undefined) {
 					self.headers[key] += ', ' + matches[2]
-				else
+				} else {
 					self.headers[key] = matches[2]
+				}
 				self.rawHeaders.push(matches[1], matches[2])
 			}
 		})
@@ -92,7 +136,15 @@ var IncomingMessage = exports.IncomingMessage = function (xhr, response, mode) {
 
 inherits(IncomingMessage, stream.Readable)
 
-IncomingMessage.prototype._read = function () {}
+IncomingMessage.prototype._read = function () {
+	var self = this
+
+	var resolve = self._resumeFetch
+	if (resolve) {
+		self._resumeFetch = null
+		resolve()
+	}
+}
 
 IncomingMessage.prototype._onXHRProgress = function () {
 	var self = this
@@ -135,7 +187,7 @@ IncomingMessage.prototype._onXHRProgress = function () {
 			}
 			break
 		case 'arraybuffer':
-			if (xhr.readyState !== rStates.DONE)
+			if (xhr.readyState !== rStates.DONE || !xhr.response)
 				break
 			response = xhr.response
 			self.push(new Buffer(new Uint8Array(response)))

package.json

@@ -1,6 +1,6 @@
 {
   "name": "stream-http",
-  "version": "2.1.1",
+  "version": "2.8.2",
   "description": "Streaming http in the browser",
   "main": "index.js",
   "repository": {
@@ -10,8 +10,8 @@
   "scripts": {
     "test": "npm run test-node && ([ -n \"${TRAVIS_PULL_REQUEST}\" -a \"${TRAVIS_PULL_REQUEST}\" != 'false' ] || npm run test-browser)",
     "test-node": "tape test/node/*.js",
-    "test-browser": "zuul --no-coverage -- test/browser/*.js",
-    "test-browser-local": "zuul --local 8080 --no-coverage -- test/browser/*.js"
+    "test-browser": "airtap --loopback airtap.local -- test/browser/*.js",
+    "test-browser-local": "airtap --no-instrument --local 8080 -- test/browser/*.js"
   },
   "author": "John Hiesey",
   "license": "MIT",
@@ -27,19 +27,20 @@
     "http-browserify"
   ],
   "dependencies": {
-    "builtin-status-codes": "^2.0.0",
+    "builtin-status-codes": "^3.0.0",
     "inherits": "^2.0.1",
+    "readable-stream": "^2.3.6",
     "to-arraybuffer": "^1.0.0",
     "xtend": "^4.0.0"
   },
   "devDependencies": {
-    "basic-auth": "^1.0.3",
-    "brfs": "^1.4.0",
-    "cookie-parser": "^1.3.5",
-    "express": "^4.13.0",
-    "tape": "^4.0.0",
-    "ua-parser-js": "^0.7.7",
-    "webworkify": "^1.0.2",
-    "zuul": "^3.1.0"
+    "airtap": "^0.0.5",
+    "basic-auth": "^2.0.0",
+    "brfs": "^1.6.1",
+    "cookie-parser": "^1.4.3",
+    "express": "^4.16.3",
+    "tape": "^4.9.0",
+    "ua-parser-js": "^0.7.18",
+    "webworkify": "^1.5.0"
   }
 }

test/browser/binary-streaming.js

@@ -8,8 +8,8 @@ var http = require('../..')
 var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
 var browserName = browser.name
 var browserVersion = browser.major
-// Binary streaming doesn't work in IE10 or below or in Opera
-var skipStreamingCheck = (browserName === 'Opera' || (browserName === 'IE' && browserVersion <= 10))
+// Binary streaming doesn't work in IE10 or below
+var skipStreamingCheck = (browserName === 'IE' && browserVersion <= 10)
 
 // Binary data gets corrupted in IE8 or below
 var skipVerification = (browserName === 'IE' && browserVersion <= 8)

test/browser/body-empty.js

@@ -0,0 +1,29 @@
+var Buffer = require('buffer').Buffer
+var fs = require('fs')
+var test = require('tape')
+
+var http = require('../..')
+
+var reference = fs.readFileSync(__dirname + '/../server/static/basic.txt')
+
+test('get body empty', function (t) {
+	var req = http.request({
+		path: '/verifyEmpty',
+		method: 'GET'
+	}, function (res) {
+		var buffers = []
+
+		res.on('end', function () {
+			console.log(Buffer.concat(buffers).toString('utf8'))
+			t.ok(Buffer.from('empty').equals(Buffer.concat(buffers)), 'response body indicates request body was empty')
+			t.end()
+		})
+
+		res.on('data', function (data) {
+			buffers.push(data)
+		})
+	})
+
+	req.write(reference)
+	req.end()
+})

test/browser/disable-fetch.js

@@ -0,0 +1,37 @@
+var Buffer = require('buffer').Buffer
+var test = require('tape')
+
+var http = require('../..')
+
+test('disable fetch', function (t) {
+	var originalFetch
+	if (typeof fetch === 'function') {
+		originalFetch = fetch
+	}
+
+	var fetchCalled = false
+	fetch = function (input, options) {
+		fetchCalled = true
+		if (originalFetch) {
+			return originalFetch(input, options)
+		}
+	}
+
+	http.get({
+		path: '/browserify.png',
+		mode: 'disable-fetch'
+	}, function (res) {
+		t.ok(!fetchCalled, 'fetch was not called')
+
+		if (originalFetch) {
+			fetch = originalFetch
+		}
+
+		res.on('end', function () {
+			t.ok(res.headers['content-type'] === 'image/png', 'content-type was set correctly')
+			t.end()
+		})
+
+		res.on('data', function () {})
+	})
+})

test/browser/headers.js

@@ -46,6 +46,40 @@ test('headers', function (t) {
 	})
 })
 
+test('arrays of headers', function (t) {
+	http.get({
+		path: '/testHeaders?Response-Header=bar&Response-Header=BAR2',
+		headers: {
+			'Test-Request-Header': ['foo', 'FOO2']
+		}
+	}, function (res) {
+		var rawHeaders = []
+		for (var i = 0; i < res.rawHeaders.length; i += 2) {
+			var lowerKey = res.rawHeaders[i].toLowerCase()
+			if (lowerKey.indexOf('test-') === 0)
+				rawHeaders.push(lowerKey, res.rawHeaders[i + 1])
+		}
+		t.equal(rawHeaders[0], 'test-response-header', 'raw response header present')
+		t.equal(rawHeaders[1], 'bar, BAR2', 'raw response header value')
+		t.equal(rawHeaders.length, 2, 'correct number of raw headers')
+
+		t.equal(res.headers['test-response-header'], 'bar, BAR2', 'response header')
+
+		var buffers = []
+
+		res.on('end', function () {
+			var body = JSON.parse(Buffer.concat(buffers).toString())
+			t.equal(body['test-request-header'], 'foo,FOO2', 'request headers')
+			t.equal(Object.keys(body).length, 1, 'correct number of request headers')
+			t.end()
+		})
+
+		res.on('data', function (data) {
+			buffers.push(data)
+		})
+	})
+})
+
 test('content-type response header', function (t) {
 	http.get('/testHeaders', function (res) {
 		t.equal(res.headers['content-type'], 'application/json', 'content-type preserved')
@@ -56,19 +90,26 @@ test('content-type response header', function (t) {
 var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
 var browserName = browser.name
 var browserVersion = browser.major
+var browserMinorVersion = browser.minor || 0
 // The content-type header is broken when 'prefer-streaming' or 'allow-wrong-content-type'
-// is passed in browsers that rely on xhr.overrideMimeType(), namely older chrome and newer safari
+// is passed in browsers that rely on xhr.overrideMimeType(), namely older chrome, safari 6-10.0, and the stock Android browser
+// Note that Safari 10.0 on iOS 10.3 doesn't need to override the mime type, so the content-type is preserved.
 var wrongMimeType = ((browserName === 'Chrome' && browserVersion <= 42) ||
-	((browserName === 'Safari' || browserName === 'Mobile Safari') && browserVersion >= 6))
+	((browserName === 'Safari' || browserName === 'Mobile Safari') && browserVersion >= 6 && (browserVersion < 10 || (browserVersion == 10 && browserMinorVersion == 0)))
+	|| (browserName === 'Android Browser'))
 
 test('content-type response header with forced streaming', function (t) {
 	http.get({
 		path: '/testHeaders',
 		mode: 'prefer-streaming'
 	}, function (res) {
-		if (wrongMimeType)
-			t.equal(res.headers['content-type'], 'text/plain; charset=x-user-defined', 'content-type overridden')
-		else
+		if (wrongMimeType) {
+			// allow both the 'wrong' and correct mime type, since sometimes it's impossible to tell which to expect
+			// from the browser version alone (e.g. Safari 10.0 on iOS 10.2 vs iOS 10.3)
+			var contentType = res.headers['content-type']
+			var correct = (contentType === 'text/plain; charset=x-user-defined') || (contentType === 'application/json')
+			t.ok(correct, 'content-type either preserved or overridden')
+		} else
 			t.equal(res.headers['content-type'], 'application/json', 'content-type preserved')
 		t.end()
 	})

test/browser/post-binary.js

@@ -9,8 +9,7 @@ var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
 var browserName = browser.name
 var browserVersion = browser.major
 // Binary request bodies don't work in a bunch of browsers
-var skipVerification = ((browserName === 'Opera' && browserVersion <= 11) ||
-	(browserName === 'IE' && browserVersion <= 10) ||
+var skipVerification = ((browserName === 'IE' && browserVersion <= 10) ||
 	(browserName === 'Safari' && browserVersion <= 5) ||
 	(browserName === 'WebKit' && browserVersion <= 534) || // Old mobile safari
 	(browserName === 'Android Browser' && browserVersion <= 4))

test/browser/text-streaming.js

@@ -8,8 +8,8 @@ var http = require('../..')
 var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
 var browserName = browser.name
 var browserVersion = browser.major
-// Streaming doesn't work in IE9 or below or in Opera
-var skipStreamingCheck = (browserName === 'Opera' || (browserName === 'IE' && browserVersion <= 9))
+// Streaming doesn't work in IE9 or below
+var skipStreamingCheck = (browserName === 'IE' && browserVersion <= 9)
 
 var COPIES = 1000
 var MIN_PIECES = 5

test/browser/text.js

@@ -1,13 +1,34 @@
 var Buffer = require('buffer').Buffer
 var fs = require('fs')
 var test = require('tape')
+var UAParser = require('ua-parser-js')
+var url = require('url')
 
 var http = require('../..')
 
+var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
+var browserName = browser.name
+var browserVersion = browser.major
+// Response urls don't work on many browsers
+var skipResponseUrl = ((browserName === 'IE') ||
+	(browserName === 'Edge') ||
+	(browserName === 'Chrome' && browserVersion <= 36) ||
+	(browserName === 'Firefox' && browserVersion <= 31) ||
+	((browserName === 'Safari' || browserName === 'Mobile Safari') && browserVersion <= 8) ||
+	(browserName === 'WebKit') || // Old mobile safari
+	(browserName === 'Android Browser' && browserVersion <= 4))
+
 var reference = fs.readFileSync(__dirname + '/../server/static/basic.txt')
 
 test('basic functionality', function (t) {
 	http.get('/basic.txt', function (res) {
+		if (!skipResponseUrl) {
+			var testUrl = url.resolve(global.location.href, '/basic.txt')
+			// Redirects aren't tested, but presumably only browser bugs
+			// would cause this to fail only after redirects.
+			t.equals(res.url, testUrl, 'response url correct')
+		}
+
 		var buffers = []
 
 		res.on('end', function () {

test/browser/timeout.js

@@ -0,0 +1,43 @@
+var Buffer = require('buffer').Buffer
+var fs = require('fs')
+var test = require('tape')
+
+var http = require('../..')
+
+test('timeout', function (t) {
+	var req = http.get({
+		path: '/browserify.png?copies=5',
+		requestTimeout: 10 // ms
+	}, function (res) {
+		res.on('data', function (data) {
+		})
+		res.on('end', function () {
+			t.fail('request completed (should have timed out)')
+		})
+	})
+	req.on('requestTimeout', function () {
+		t.pass('got timeout')
+		t.end()
+	})
+})
+
+// TODO: reenable this if there's a way to make it simultaneously
+// fast and reliable
+test.skip('no timeout after success', function (t) {
+	var req = http.get({
+		path: '/basic.txt',
+		requestTimeout: 50000 // ms
+	}, function (res) {
+		res.on('data', function (data) {
+		})
+		res.on('end', function () {
+			t.pass('success')
+			global.setTimeout(function () {
+				t.end()
+			}, 50000)
+		})
+	})
+	req.on('requestTimeout', function () {
+		t.fail('unexpected timeout')
+	})
+})

test/browser/webworker.js

@@ -8,8 +8,7 @@ var browser = (new UAParser()).setUA(navigator.userAgent).getBrowser()
 var browserName = browser.name
 var browserVersion = browser.major
 // Skip browsers with poor or nonexistant WebWorker support
-var skip = ((browserName === 'Opera' && browserVersion <= 12) ||
-	(browserName === 'IE' && browserVersion <= 10) ||
+var skip = ((browserName === 'IE' && browserVersion <= 10) ||
 	(browserName === 'Safari' && browserVersion <= 5) ||
 	(browserName === 'WebKit' && browserVersion <= 534) || // Old mobile safari
 	(browserName === 'Android Browser' && browserVersion <= 4))

test/node/http-browserify.js

@@ -1,4 +1,4 @@
-// These tests are teken from http-browserify to ensure compatibility with
+// These tests are taken from http-browserify to ensure compatibility with
 // that module
 var test = require('tape')
 var url = require('url')
@@ -17,6 +17,20 @@ var moduleName = require.resolve('../../')
 delete require.cache[moduleName]
 var http = require('../../')
 
+test('Make sure http object has correct properties', function (t) {
+	t.ok(http.Agent, 'Agent defined')
+	t.ok(http.ClientRequest, 'ClientRequest defined')
+	t.ok(http.ClientRequest.prototype, 'ClientRequest.prototype defined')
+	t.ok(http.IncomingMessage, 'IncomingMessage defined')
+	t.ok(http.IncomingMessage.prototype, 'IncomingMessage.prototype defined')
+	t.ok(http.METHODS, 'METHODS defined')
+	t.ok(http.STATUS_CODES, 'STATUS_CODES defined')
+	t.ok(http.get, 'get defined')
+	t.ok(http.globalAgent, 'globalAgent defined')
+	t.ok(http.request, 'request defined')
+	t.end()
+})
+
 test('Test simple url string', function(t) {
 	var testUrl = { path: '/api/foo' }
 	var request = http.get(testUrl, noop)

test/server/index.js

@@ -32,13 +32,17 @@ app.get('/testHeaders', function (req, res) {
 	})
 
 	res.setHeader('Content-Type', 'application/json')
+	res.setHeader('Cache-Control', 'no-cache')
 
 	// Request headers are sent in the body as json
 	var reqHeaders = {}
 	Object.keys(req.headers).forEach(function (key) {
 		key = key.toLowerCase()
-		if (key.indexOf('test-') === 0)
-			reqHeaders[key] = req.headers[key]
+		if (key.indexOf('test-') === 0) {
+			// different browsers format request headers with multiple values
+			// slightly differently, so normalize
+			reqHeaders[key] = req.headers[key].replace(', ', ',')
+		}
 	})
 
 	var body = JSON.stringify(reqHeaders)
@@ -71,6 +75,24 @@ app.post('/echo', function (req, res) {
 	req.pipe(res)
 })
 
+app.use('/verifyEmpty', function (req, res) {
+	var empty = true
+	req.on('data', function (buf) {
+		if (buf.length > 0) {
+			empty = false
+		}
+	})
+	req.on('end', function () {
+		res.setHeader('Content-Type', 'text/plain')
+
+		if (empty) {
+			res.end('empty')
+		} else {
+			res.end('not empty')
+		}
+	})
+})
+
 app.use(function (req, res, next) {
 	var parsed = url.parse(req.url, true)
 
@@ -110,6 +132,6 @@ app.use(function (req, res, next) {
 
 app.use(express.static(path.join(__dirname, 'static')))
 
-var port = parseInt(process.env.ZUUL_PORT) || 8199
+var port = parseInt(process.env.AIRTAP_PORT) || 8199
 console.log('Test server listening on port', port)
 server.listen(port)