webunit-1.3.10/000755 000765 000024 00000000000 11370200213 014015 5ustar00richardstaff000000 000000 webunit-1.3.10/CHANGES.txt000777 000765 000024 00000003200 11370177631 015651 0ustar00richardstaff000000 000000 2010-05-05 1.3.10 - fix cookie domain matching to allow cookie '.domain.com' to match host 'domain.com' 2007-03-26 1.3.9 - send \r\n in mimeEncode (thanks Ivan Kurmanov) - handle Max-Age set to 0 (thanks Matt Chisholm) 2004-01-30 1.3.8 - fix syntax error ;) 2004-01-22 1.3.7 - fix use of radio/checkbox/select arguments in constructing GET URLs (thanks Jeroen Vloothuis and Alexandre Fayolle) - corrected detection of closing tag for CDATA 2004-01-21 1.3.6 - remove debug print - propogate error content - added getById(name, id) to SimpleDOM objects, allowing retrieval by tag type (eg. div) and id (ie.
) 2003-11-03 1.3.5 - patch from Diez B. Roggisch to handle headers.get('set-cookie', '') returning comma-separated list of Set-Cookies - really changed SmartCookie to SimpleCookie - handle request path == '' but cookie path == '/' - fixes to base url handling - added method clearContext() to clear "browser state" so setUp cookies/auth info/images/base url etc. don't affect tests if you don't want them to 2003-10-29 1.3.4 - change older httpslib.HTTPS (M2Crypto) usage to python's httplib.HTTPS 2003-10-10 1.3.3 - change use of SmartCookie to SimpleCookie 2003-10-08 1.3.2 - fixed some bugs here and there 2003-09 1.3.1 - fixed a bunch of bugs to do with not honoring the tag 2003-07 1.3 - packaging with distutils now - fixed following of relative urls in requests on HTTPResponse objects 2003-06-16 1.2.2 - help the WebFetcher cope with '.' form actions (thanks Roché Compaan) 2003-05-17 1.2.1 - wasn't creating correct DOM node type in SimpleDOM (thanks Roché Compaan) webunit-1.3.10/demo/000755 000765 000024 00000000000 11370200213 014741 5ustar00richardstaff000000 000000 webunit-1.3.10/PKG-INFO000644 000765 000024 00000003554 11370200213 015121 0ustar00richardstaff000000 000000 Metadata-Version: 1.0 Name: webunit Version: 1.3.10 Summary: Unit test your websites with code that acts like a web browser. Home-page: http://mechanicalcat.net/tech/webunit/ Author: Richard Jones Author-email: richard@mechanicalcat.net License: UNKNOWN Download-URL: http://pypi.python.org/pypi/webunit Description: This release includes: - send correct newline in mimeEncode (thanks Ivan Kurmanov) - handle Max-Age set to 0 (thanks Matt Chisholm) Webunit is a framework for unit testing websites: - Browser-like page fetching including fetching the images and stylesheets needed for a page and following redirects - Cookies stored and trackable (all automatically handled) - HTTP, HTTPS, GET, POST, basic auth all handled, control over expected status codes, ... - DOM parsing of pages to retrieve and analyse structure, including simple form re-posting - Two-line page-fetch followed by form-submit possible, with error checking - Ability to register error page content across multiple tests - Uses python's standard unittest module as the underlying framework - May also be used to regression-test sites, or ensure their ongoing operation once in production (testing login processes work, etc.) Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Environment :: Console Classifier: Intended Audience :: Developers Classifier: Intended Audience :: System Administrators Classifier: Natural Language :: English Classifier: License :: OSI Approved :: BSD License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Topic :: Internet :: WWW/HTTP :: Site Management Classifier: Topic :: Software Development :: Testing Classifier: Topic :: System :: Monitoring webunit-1.3.10/README.txt000777 000765 000024 00000021776 11370177720 015557 0ustar00richardstaff000000 000000 Web unit testing (the concise help) ==================================== To run the current demo tests, use:: ./run_tests demo Installation ------------ Install the webunit libraries with:: python setup.py install Then make a directory like the demo one in your application with tests in it, and invoke with:: ./run_tests Run the test named - just like unittest.py. Configuration ------------- We've found it useful to be able to test different servers with the same test suite. To this end, we've got a simple configuration file setup. See the doc string in webunit.config Errors ------ If a request fails with an incorrect response code (the default valid response codes are 200, 301 and 302) then the result body fetched from the server will be appended to the logfile for that server. Making requests - the WebTestCase class --------------------------------------- The WebTestCase is best thought of as a web browser with some added features. To truly emulate a web browser (single-threaded at present) use the "page" methods below. For more fine-grained control and testing, use the other methods. The WebTestCase objects have a number of attributes (see `Setting up fetch defaults`_): 1. protocol, server, port -- these default to http://localhost:80 and are used when a relative fetch is performed. Most tests set these vars up with setServer in the setUp method and then use relative URLs. 2. authinfo -- basic authentication information. Use setBasicAuth to set and clearBasicAuth to clear. 3. cookies -- the test's store of cookies 4. images -- cache of images fetched There are two modes of retrieval using two HTTP methods: **fetch** mode This is the default mode, and just fetches the HTML of a page. **page** mode This is a more browser-like mode. It follows redirections and loads images and stylesheets required to render the page. **GET** method This is the default HTTP method used. **POST** method All GET methods have a HTTP POST analog, usually just with ``post`` prefixed. Now the actual calls that are possible: assertCode(self, url, code=None, \*\*kw) Perform a HTTP GET and assert that the return code from the server one of the indicated codes. get = assertCode(self, url, code=None, \*\*kw) Just an alias assertContent(self, url, content, code=None, \*\*kw) Perform a HTTP GET and assert that the data returned from the server contains the indicated content string. assertNotContent(self, url, content, code=None, \*\*kw) Perform a HTTP GET and assert that the data returned from the server contains the indicated content string. getAssertCode = assertCode(self, url, code=None, \*\*kw) Just an alias getAssertContent = assertContent(self, url, content, code=None, \*\*kw) Just an alias getAssertNotContent = assertNotContent(self, url, content, code=None, \*\*kw) Just an alias page(self, url, code=None, \*\*kw) Perform a HTTP GET using the specified URL and then retrieve all image and linked stylesheet components for the resulting HTML page. post(self, url, params, code=None, \*\*kw) Perform a HTTP POST using the specified URL and form parameters. postAssertCode(self, url, params, code=None, \*\*kw) Perform a HTTP POST and assert that the return code from the server is one of the indicated codes. postAssertContent(self, url, params, content, code=None, \*\*kw) Perform a HTTP POST and assert that the data returned from the server contains the indicated content string. postAssertNotContent(self, url, params, content, code=None, \*\*kw) Perform a HTTP POST and assert that the data returned from the server doesn't contain the indicated content string. postPage(self, url, params, code=None, \*\*kw) Perform a HTTP POST using the specified URL and form parameters and then retrieve all image and linked stylesheet components for the resulting HTML page. All of these methods eventually call fetch() - the additional \*\*kw are passed directly to the fetch method: fetch(self, url, postdata=None, server=None, port=None, protocol=None, ok_codes=None) Run a single test request to the indicated url. Use the POST data if supplied. If the URL is a fully-qualified one (ie. has a server and protocol) then that overrides the session's default, but may be further overridden by the method arguments. Raises failureException if the returned data contains any of the strings indicated to be Error Content. Returns a HTTPReponse object wrapping the response from the server. HTTP Response objects --------------------- The HTTPResponse objects hold all the infomation about the server's response to the request. This information includes: 1. protocol, server, port, url - the request server and URL 2. code, message, headers - the information returned by httplib.HTTP.getreply() 3. body - the response body returned by httplib.HTTP.getfile() Additionally, the object has several methods: getDOM(self) Get a DOM for this page. See the SimpleDOM_ instructions for details extractForm(self, path=[], include_submit=0, include_button=0) Extract a form (as a dictionary) from this page. The "path" is a list of 2-tuples ``('element name', index)`` to follow to find the form. So:: ..

...

...

To extract the second form, any of these could be used:: [('html',0), ('body',0), ('p',1), ('form',0)] [('form',1)] [('p',1)] HTTPResponse objects are also able to fetch using WebTestCase methods. They define additional methods: getForm(self, formnum, getmethod, postargs, \*args) Given this page, extract the "formnum"th form from it, fill the form with the "postargs" and post back to the server using the "getmethod" with additional "args". NOTE: the form submission will include any "default" values from the form extracted from this page. To "remove" a value from the form, just pass a value None for the elementn and it will be removed from the form submission. example WebTestCase:: page = self.get('/foo') page.postForm(0, self.post, {'name': 'blahblah', 'password': 'foo'}) or the slightly more complex:: page = self.get('/foo') page.postForm(0, self.postAssertContent, {'name': 'blahblah', 'password': None}, 'password incorrect') postForm(self, formnum, postmethod, postargs, \*args) As with getForm, only use a POST request. Setting up fetch defaults ------------------------- setServer(self, server, port) Set the server and port number to perform the HTTP requests to. setBasicAuth(self, username, password) Set the Basic authentication information to the given username and password. clearBasicAuth(self) Clear the current Basic authentication information setAcceptCookies(self, accept=1) Indicate whether to accept cookies or not clearCookies(self) Clear all currently received cookies Auto-fail error content ----------------------- You may have the fetcher automatically fail when receiving certain content using: registerErrorContent(self, content) Register the given string as content that should be considered a test failure (even though the response code is 200). removeErrorContent(self, content) Remove the given string from the error content list. clearErrorContent(self) Clear the current list of error content strings. Cookies ------- To test for cookies being sent _to_ a server, use: registerExpectedCookie(self, cookie) Register a cookie name that we expect to send to the server. removeExpectedCookie(self, cookie) Remove the given cookie from the list of cookies we expect to send to the server. To test for cookies sent _from_ the server, access the cookies attribute of the test harness. It is a dict of:: cookies[host name][path][cookie name] = string SimpleDOM --------- Simple usage:: >>> import SimpleDOM >>> parser = SimpleDOM.SimpleDOMParser() >>> parser.parseString("""My Document ... ...

This is a paragraph!!!

...

This is another para!!

... ... """) >>> dom = parser.getDOM() >>> dom.getByName('p') [, ] >>> dom.getByName('p')[0][0] 'This is a paragraph!!!' >>> dom.getByName('title')[0][0] 'My Document' Form extraction example (see also the tests in the test/ directory of the source):: # fetch the start page page = self.get('/ekit/home') page = page.postForm(1, self.postAssertCode, {'__ac_name': 'joebloggs', '__ac_password': 'foo'}, [302]) # same as last fetch, but automatically follow the redirect page = page.postForm(1, self.page, {'__ac_name': 'joebloggs', '__ac_password': 'foo'}) Thanks ====== Thanks to everyone who's helped with this package, including supplying hints, bug reports and (more importantly :) patches: Gary Capell Note that this list is nowhere near complete, as I've only just started maintaining it. If you're miffed that you're not on it, just let me know! webunit-1.3.10/run_tests000777 000765 000024 00000001275 11212244064 016011 0ustar00richardstaff000000 000000 #! /usr/bin/env python import unittest, sys, types class TestLoader(unittest.TestLoader): def loadTestsFromModule(self, module): '''Override so we can use suites already defined in the modules ''' tests = [] if hasattr(module, 'suite'): return module.suite() for name in dir(module): obj = getattr(module, name) if type(obj) == types.ClassType and issubclass(obj, TestCase): tests.append(self.loadTestsFromTestCase(obj)) return self.suiteClass(tests) if __name__ == '__main__': unittest.TestProgram(module=None, argv=sys.argv, testLoader=TestLoader()) # vim: set filetype=python ts=4 sw=4 et si webunit-1.3.10/setup.py000777 000765 000024 00000004017 11341375616 015562 0ustar00richardstaff000000 000000 #! /usr/bin/env python # # $Id: setup.py,v 1.9 2004/01/21 22:51:40 richard Exp $ from distutils.core import setup # perform the setup action from webunit import __version__ setup( name = "webunit", version = __version__, description = "Unit test your websites with code that acts like a web browser.", long_description = '''This release includes: - send correct newline in mimeEncode (thanks Ivan Kurmanov) - handle Max-Age set to 0 (thanks Matt Chisholm) Webunit is a framework for unit testing websites: - Browser-like page fetching including fetching the images and stylesheets needed for a page and following redirects - Cookies stored and trackable (all automatically handled) - HTTP, HTTPS, GET, POST, basic auth all handled, control over expected status codes, ... - DOM parsing of pages to retrieve and analyse structure, including simple form re-posting - Two-line page-fetch followed by form-submit possible, with error checking - Ability to register error page content across multiple tests - Uses python's standard unittest module as the underlying framework - May also be used to regression-test sites, or ensure their ongoing operation once in production (testing login processes work, etc.) ''', author = "Richard Jones", author_email = "richard@mechanicalcat.net", url = 'http://mechanicalcat.net/tech/webunit/', download_url = 'http://pypi.python.org/pypi/webunit', packages = ['webunit', 'demo'], classifiers = [ 'Development Status :: 5 - Production/Stable', 'Environment :: Console', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', 'Natural Language :: English', 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Topic :: Internet :: WWW/HTTP :: Site Management', 'Topic :: Software Development :: Testing', 'Topic :: System :: Monitoring', ], ) # vim: set filetype=python ts=4 sw=4 et si webunit-1.3.10/webunit/000755 000765 000024 00000000000 11370200213 015472 5ustar00richardstaff000000 000000 webunit-1.3.10/webunit/__init__.py000777 000765 000024 00000000027 11370177710 017630 0ustar00richardstaff000000 000000 __version__ = '1.3.10' webunit-1.3.10/webunit/config.py000777 000765 000024 00000001306 11212244064 017327 0ustar00richardstaff000000 000000 ''' This file allows you to set up configuration variables to identify the machine and port to test. It needs some work, but in a nutshell, put a config.cfg in your "test" directory with the following contents: ---- snip [DEFAULT] machine = www.dev.ekorp.com port = 80 [dev-ekit] # uses DEFAULT [dev-lp] machine = www.lonelyplanet.dev.ekorp.com port = 80 ---- snip Then set the environment var "TEST_CONFIG" to the config to use. ''' import ConfigParser, os cfg = ConfigParser.ConfigParser() cfg.read('test/config.cfg') # figure the active config active = os.environ.get('TEST_CONFIG', 'DEFAULT') # fetch the actual config info machine = cfg.get(active, 'machine') port = cfg.getint(active, 'port') webunit-1.3.10/webunit/cookie.py000777 000765 000024 00000010005 11370200117 017323 0ustar00richardstaff000000 000000 import re, urlparse, Cookie class Error: '''Handles a specific cookie error. message - a specific message as to why the cookie is erroneous ''' def __init__(self, message): self.message = str(message) def __str__(self): return 'COOKIE ERROR: %s'%self.message def parse_cookie(text, qparmre=re.compile( r'([\0- ]*([^\0- ;,=\"]+)="([^"]*)\"([\0- ]*[;,])?[\0- ]*)'), parmre=re.compile( r'([\0- ]*([^\0- ;,=\"]+)=([^\0- ;,\"]*)([\0- ]*[;,])?[\0- ]*)')): result = {} l = 0 while 1: if qparmre.match(text[l:]) >= 0: # Match quoted correct cookies name=qparmre.group(2) value=qparmre.group(3) l=len(qparmre.group(1)) elif parmre.match(text[l:]) >= 0: # Match evil MSIE cookies ;) name=parmre.group(2) value=parmre.group(3) l=len(parmre.group(1)) else: # this may be an invalid cookie. # We'll simply bail without raising an error # if the cookie is invalid. return result if not result.has_key(name): result[name]=value return result def decodeCookies(url, server, headers, cookies): '''Decode cookies into the supplied cookies dictionary Relevant specs: http://www.ietf.org/rfc/rfc2109.txt http://www.ietf.org/rfc/rfc2965.txt ''' # the path of the request URL up to, but not including, the right-most / request_path = urlparse.urlparse(url)[2] if len(request_path) > 1 and request_path[-1] == '/': request_path = request_path[:-1] hdrcookies = Cookie.SimpleCookie("\n".join(map(lambda x: x.strip(), headers.getallmatchingheaders('set-cookie')))) for cookie in hdrcookies.values(): # XXX: there doesn't seem to be a way to determine if the # cookie was set or defaulted to an empty string :( if cookie['domain']: domain = cookie['domain'] # reject if The value for the Domain attribute contains no # embedded dots or does not start with a dot. if '.' not in domain: raise Error, 'Cookie domain "%s" has no "."'%domain if domain[0] != '.': # per RFC2965 cookie domains with no leading '.' will have # one added domain = '.' + domain # reject if the value for the request-host does not # domain-match the Domain attribute. # For cookie .example.com we should allow: # - example.com # - www.example.com # but not: # - someexample.com if not server.endswith(domain) and domain[1:] != server: raise Error, 'Cookie domain "%s" doesn\'t match '\ 'request host "%s"'%(domain, server) # reject if the request-host is a FQDN (not IP address) and # has the form HD, where D is the value of the Domain # attribute, and H is a string that contains one or more dots. if re.search(r'[a-zA-Z]', server): H = server[:-len(domain)] if '.' in H: raise Error, 'Cookie domain "%s" too short '\ 'for request host "%s"'%(domain, server) else: domain = server # path check path = cookie['path'] or request_path # reject if Path attribute is not a prefix of the request-URI # (noting that empty request path and '/' are often synonymous, yay) if not (request_path.startswith(path) or (request_path == '' and cookie['path'] == '/')): raise Error, 'Cookie path "%s" doesn\'t match '\ 'request url "%s"'%(path, request_path) bydom = cookies.setdefault(domain, {}) bypath = bydom.setdefault(path, {}) maxage = cookie.get('max-age', '1') if maxage != '0': bypath[cookie.key] = cookie elif cookie.key in bypath: del bypath[cookie.key] webunit-1.3.10/webunit/HTMLParser.py000777 000765 000024 00000036554 11212244064 020020 0ustar00richardstaff000000 000000 """A parser for HTML.""" # This file is derived from sgmllib.py, which is part of Python. # XXX There should be a way to distinguish between PCDATA (parsed # character data -- the normal case), RCDATA (replaceable character # data -- only char and entity references and end tags are special) # and CDATA (character data -- only end tags are special). import re import string # Regular expressions used for parsing interesting_normal = re.compile('[&<]') interesting_cdata = re.compile(r'<(/|\Z)') incomplete = re.compile('(&[a-zA-Z][-.a-zA-Z0-9]*|&#[0-9]*)') entityref = re.compile('&([a-zA-Z][-.a-zA-Z0-9]*)[^a-zA-Z0-9]') charref = re.compile('&#([0-9]+)[^0-9]') starttagopen = re.compile('<[a-zA-Z]') piopen = re.compile(r'<\?') piclose = re.compile('>') endtagopen = re.compile(']*>') commentopen = re.compile('" % data) def handle_decl(self, data): self.emitText("" % data) def handle_pi(self, data): self.emitText("" % data) def emitStartTag(self, name, attrlist, isend=0): if isend: if self.__debug: print '*** content' self.content.append(SimpleDOMNode(name, attrlist, [])) else: # generate a new scope and push the current one on the stack if self.__debug: print '*** push' newcontent = [] self.stack.append(self.content) self.content.append(SimpleDOMNode(name, attrlist, newcontent)) self.content = newcontent def emitEndTag(self, name): if self.__debug: print '*** pop' self.content = self.stack.pop() def emitText(self, text): self.content.append(text) def emitStartElement(self, name, attrlist, isend=0): # Handle the simple, common case self.emitStartTag(name, attrlist, isend) if isend: self.emitEndElement(name, isend) def emitEndElement(self, name, isend=0, implied=0): if not isend or implied: self.emitEndTag(name) if __name__ == '__main__': tester = SimpleDOMParser(debug=0) tester.parseFile('/tmp/test.html') dom = tester.getDOM() # html = dom.getByNameFlat('html')[0] # body = html.getByNameFlat('body')[0] # table = body.getByNameFlat('table')[0] # tr = table.getByNameFlat('tr')[1] # td = tr.getByNameFlat('td')[2] # print td import pprint;pprint.pprint(dom) webunit-1.3.10/webunit/utility.py000777 000765 000024 00000005100 11212244064 017561 0ustar00richardstaff000000 000000 # # Copyright (c) 2003 Richard Jones (http://mechanicalcat.net/richard) # Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/) # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) # # See the README for full license details. # # $Id$ import cStringIO import os.path class Upload: '''Simple "sentinel" class that lets us identify file uploads in POST data mappings. ''' def __init__(self, filename): self.filename = filename def __cmp__(self, other): return cmp(self.filename, other.filename) boundary = '--------------GHSKFJDLGDS7543FJKLFHRE75642756743254' sep_boundary = '\n--' + boundary end_boundary = sep_boundary + '--' def mimeEncode(data, sep_boundary=sep_boundary, end_boundary=end_boundary): '''Take the mapping of data and construct the body of a multipart/form-data message with it using the indicated boundaries. ''' ret = cStringIO.StringIO() for key, value in data.items(): if not key: continue # handle multiple entries for the same name if type(value) != type([]): value = [value] for value in value: ret.write(sep_boundary) # if key starts with a '$' then the entry is a file upload if isinstance(value, Upload): ret.write('\r\nContent-Disposition: form-data; name="%s"'%key) ret.write('; filename="%s"\n\n'%value.filename) if value.filename: value = open(os.path.join(value.filename), "rb").read() else: value = '' else: ret.write('\r\nContent-Disposition: form-data; name="%s"'%key) ret.write("\r\n\r\n") ret.write(str(value)) if value and value[-1] == '\r': ret.write('\n') # write an extra newline ret.write(end_boundary) return ret.getvalue() def log(message, content, logfile='logfile'): '''Log a single message to the indicated logfile ''' logfile = open(logfile, 'a') logfile.write('\n>>> %s\n'%message) logfile.write(str(content) + '\n') logfile.close() # # $Log$ # Revision 1.3 2003/08/23 02:01:59 richard # fixes to cookie sending # # Revision 1.2 2003/07/22 01:19:22 richard # patches # # Revision 1.1.1.1 2003/07/22 01:01:44 richard # # # Revision 1.4 2002/02/25 02:59:09 rjones # *** empty log message *** # # Revision 1.3 2002/02/22 06:24:31 rjones # Code cleanup # # Revision 1.2 2002/02/13 01:16:56 rjones # *** empty log message *** # # # vim: set filetype=python ts=4 sw=4 et si webunit-1.3.10/webunit/webunittest.py000777 000765 000024 00000062744 11212244064 020454 0ustar00richardstaff000000 000000 # # Copyright (c) 2003 Richard Jones (http://mechanicalcat.net/richard) # Copyright (c) 2002 ekit.com Inc (http://www.ekit-inc.com/) # Copyright (c) 2001 Bizar Software Pty Ltd (http://www.bizarsoftware.com.au/) # # See the README for full license details. # # $Id: webunittest.py,v 1.13 2004/08/26 02:50:19 richard Exp $ import os, base64, urllib, urlparse, unittest, cStringIO, time, re, sys import httplib #try: # from M2Crypto import httpslib #except ImportError: # httpslib = None from SimpleDOM import SimpleDOMParser from IMGSucker import IMGSucker from utility import Upload, mimeEncode, boundary, log import cookie VERBOSE = os.environ.get('VERBOSE', '') class HTTPError: '''Wraps a HTTP response that is not 200. url - the URL that generated the error code, message, headers - the information returned by httplib.HTTP.getreply() ''' def __init__(self, response): self.response = response def __str__(self): return 'ERROR: %s'%str(self.response) class WebFetcher: '''Provide a "web client" class that handles fetching web pages. Handles basic authentication, HTTPS, detection of error content, ... Creates a HTTPResponse object on a valid response. Stores cookies received from the server. ''' def __init__(self): '''Initialise the server, port, authinfo, images and error_content attributes. ''' self.protocol = 'http' self.server = '127.0.0.1' self.port = 80 self.authinfo = '' self.url = None self.images = {} self.error_content = [] self.expect_codes = [200, 301, 302] self.expect_content = None self.expect_cookies = None self.accept_cookies = 1 self.debug_headers = 0 self.cookies = {} result_count = 0 def clearContext(self): self.authinfo = '' self.cookies = {} self.url = None self.images = {} def setServer(self, server, port): '''Set the server and port number to perform the HTTP requests to. ''' self.server = server self.port = int(port) # # Authentication # def clearBasicAuth(self): '''Clear the current Basic authentication information ''' self.authinfo = '' def setBasicAuth(self, username, password): '''Set the Basic authentication information to the given username and password. ''' self.authinfo = base64.encodestring('%s:%s'%(username, password)).strip() # # cookie handling # def clearCookies(self): '''Clear all currently received cookies ''' self.cookies = {} def setAcceptCookies(self, accept=1): '''Indicate whether to accept cookies or not ''' self.accept_cookies = accept def registerErrorContent(self, content): '''Register the given string as content that should be considered a test failure (even though the response code is 200). ''' self.error_content.append(content) def removeErrorContent(self, content): '''Remove the given string from the error content list. ''' self.error_content.remove(content) def clearErrorContent(self): '''Clear the current list of error content strings. ''' self.error_content = [] def log(self, message, content): '''Log a message to the logfile ''' log(message, content, 'logfile.'+self.server) # # Register cookies we expect to send to the server # def registerExpectedCookie(self, cookie): '''Register a cookie name that we expect to send to the server. ''' if self.expect_cookies is None: self.expect_cookies = [cookie] return self.expect_cookies.append(cookie) self.expect_cookies.sort() def removeExpectedCookie(self, cookie): '''Remove the given cookie from the list of cookies we expect to send to the server. ''' self.expect_cookies.remove(cookie) def clearExpectedCookies(self): '''Clear the current list of cookies we expect to send to the server. ''' self.expect_cookies = None # # POST # def post(self, url, params, code=None, **kw): '''Perform a HTTP POST using the specified URL and form parameters. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 try: response = self.fetch(url, params, ok_codes=code, **kw) except HTTPError, error: self.log('post'+`(url, params)`, error.response.body) raise self.failureException, str(error.response) return response def postAssertCode(self, url, params, code=None, **kw): '''Perform a HTTP POST and assert that the return code from the server is one of the indicated codes. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 if type(code) != type([]): code = [code] try: response = self.fetch(url, params, ok_codes = code, **kw) except HTTPError, error: self.log('postAssertCode'+`(url, params, code)`, error.response.body) raise self.failureException, str(error.response) return response def postAssertContent(self, url, params, content, code=None, **kw): '''Perform a HTTP POST and assert that the data returned from the server contains the indicated content string. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 if type(code) != type([]): code = [code] try: response = self.fetch(url, params, ok_codes = code, **kw) except HTTPError, error: self.log('postAssertContent'+`(url, params, code)`, error.response.body) raise self.failureException, str(error) if response.body.find(content) == -1: self.log('postAssertContent'+`(url, params, content)`, response.body) raise self.failureException, 'Expected content not in response' return response def postAssertNotContent(self, url, params, content, code=None, **kw): '''Perform a HTTP POST and assert that the data returned from the server doesn't contain the indicated content string. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 if type(code) != type([]): code = [code] try: response = self.fetch(url, params, ok_codes = code, **kw) except HTTPError, error: self.log('postAssertNotContent'+`(url, params, code)`, error.response.body) raise self.failureException, str(error) if response.body.find(content) != -1: self.log('postAssertNotContent'+`(url, params, content)`, response.body) raise self.failureException, 'Expected content was in response' return response def postPage(self, url, params, code=None, **kw): '''Perform a HTTP POST using the specified URL and form parameters and then retrieve all image and linked stylesheet components for the resulting HTML page. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 try: response = self.fetch(url, params, ok_codes=code, **kw) except HTTPError, error: self.log('postPage %r'%((url, params),), error.response.body) raise self.failureException, str(error) # Check return code for redirect while response.code in (301, 302): try: # Figure the location - which may be relative newurl = response.headers['Location'] url = urlparse.urljoin(url, newurl) response = self.fetch(url, ok_codes=code) except HTTPError, error: self.log('postPage %r'%url, error.response.body) raise self.failureException, str(error) # read and parse the content page = response.body if hasattr(self, 'results') and self.results: self.writeResult(url, page) try: self.pageImages(url, page) except HTTPError, error: raise self.failureException, str(error) return response # # GET # def assertCode(self, url, code=None, **kw): '''Perform a HTTP GET and assert that the return code from the server one of the indicated codes. ''' if code is None: code = self.expect_codes return self.postAssertCode(url, None, code=code, **kw) get = getAssertCode = assertCode def assertContent(self, url, content, code=None, **kw): '''Perform a HTTP GET and assert that the data returned from the server contains the indicated content string. ''' if code is None: code = self.expect_codes return self.postAssertContent(url, None, content, code) getAssertContent = assertContent def assertNotContent(self, url, content, code=None, **kw): '''Perform a HTTP GET and assert that the data returned from the server contains the indicated content string. ''' if code is None: code = self.expect_codes return self.postAssertNotContent(url, None, content, code) getAssertNotContent = assertNotContent def page(self, url, code=None, **kw): '''Perform a HTTP GET using the specified URL and then retrieve all image and linked stylesheet components for the resulting HTML page. ''' if code is None: code = self.expect_codes WebTestCase.result_count = WebTestCase.result_count + 1 return self.postPage(url, None, code=code, **kw) def get_base_url(self): # try to get a tag and use that to root the URL on if hasattr(self, 'getDOM'): base = self.getDOM().getByName('base') if base: # return base[0].href if self.url is not None: # join the request URL with the "current" URL return self.url return None # # The function that does it all # def fetch(self, url, postdata=None, server=None, port=None, protocol=None, ok_codes=None): '''Run a single test request to the indicated url. Use the POST data if supplied. Raises failureException if the returned data contains any of the strings indicated to be Error Content. Returns a HTTPReponse object wrapping the response from the server. ''' # see if the url is fully-qualified (not just a path) t_protocol, t_server, t_url, x, t_args, x = urlparse.urlparse(url) if t_server: protocol = t_protocol if ':' in t_server: server, port = t_server.split(':') else: server = t_server if protocol == 'http': port = '80' else: port = '443' url = t_url if t_args: url = url + '?' + t_args # ignore the machine name if the URL is for localhost if t_server == 'localhost': server = None elif not server: # no server was specified with this fetch, or in the URL, so # see if there's a base URL to use. base = self.get_base_url() if base: t_protocol, t_server, t_url, x, x, x = urlparse.urlparse(base) if t_protocol: protocol = t_protocol if t_server: server = t_server if t_url: url = urlparse.urljoin(t_url, url) # TODO: allow override of the server and port from the URL! if server is None: server = self.server if port is None: port = self.port if protocol is None: protocol = self.protocol if ok_codes is None: ok_codes = self.expect_codes if protocol == 'http': h = httplib.HTTP(server, int(port)) if int(port) == 80: host_header = server else: host_header = '%s:%s'%(server, port) elif protocol == 'https': #if httpslib is None: #raise ValueError, "Can't fetch HTTPS: M2Crypto not installed" h = httplib.HTTPS(server, int(port)) if int(port) == 443: host_header = server else: host_header = '%s:%s'%(server, port) else: raise ValueError, protocol headers = [] params = None if postdata: for field,value in postdata.items(): if type(value) == type({}): postdata[field] = [] for k,selected in value.items(): if selected: postdata[field].append(k) # Do a post with the data file params = mimeEncode(postdata) h.putrequest('POST', url) headers.append(('Content-type', 'multipart/form-data; boundary=%s'%boundary)) headers.append(('Content-length', str(len(params)))) else: # Normal GET h.putrequest('GET', url) # Other Full Request headers if self.authinfo: headers.append(('Authorization', "Basic %s"%self.authinfo)) headers.append(('Host', host_header)) # Send cookies # - check the domain, max-age (seconds), path and secure # (http://www.ietf.org/rfc/rfc2109.txt) cookies_used = [] cookie_list = [] for domain, cookies in self.cookies.items(): # check cookie domain if not server.endswith(domain): continue for path, cookies in cookies.items(): # check that the path matches urlpath = urlparse.urlparse(url)[2] if not urlpath.startswith(path) and not (path == '/' and urlpath == ''): continue for sendcookie in cookies.values(): # and that the cookie is or isn't secure if sendcookie['secure'] and protocol != 'https': continue # TODO: check max-age cookie_list.append("%s=%s;"%(sendcookie.key, sendcookie.coded_value)) cookies_used.append(sendcookie.key) if cookie_list: headers.append(('Cookie', ' '.join(cookie_list))) # check that we sent the cookies we expected to if self.expect_cookies is not None: assert cookies_used == self.expect_cookies, \ "Didn't use all cookies (%s expected, %s used)"%( self.expect_cookies, cookies_used) # write and finish the headers for header in headers: h.putheader(*header) h.endheaders() if self.debug_headers: import pprint;pprint.pprint(headers) if params is not None: h.send(params) # handle the reply errcode, errmsg, headers = h.getreply() # get the body and save it f = h.getfile() g = cStringIO.StringIO() d = f.read() while d: g.write(d) d = f.read() response = HTTPResponse(self.cookies, protocol, server, port, url, errcode, errmsg, headers, g.getvalue(), self.error_content) f.close() if errcode not in ok_codes: if VERBOSE: sys.stdout.write('e') sys.stdout.flush() raise HTTPError(response) # decode the cookies if self.accept_cookies: try: # decode the cookies and update the cookies store cookie.decodeCookies(url, server, headers, self.cookies) except: if VERBOSE: sys.stdout.write('c') sys.stdout.flush() raise # Check errors if self.error_content: data = response.body for content in self.error_content: if data.find(content) != -1: url = urlparse.urlunparse((protocol, server, url, '','','')) msg = "URL %r matched error: %s"%(url, content) if hasattr(self, 'results') and self.results: self.writeError(url, msg) self.log('Matched error'+`(url, content)`, data) if VERBOSE: sys.stdout.write('c') sys.stdout.flush() raise self.failureException, msg if VERBOSE: sys.stdout.write('_') sys.stdout.flush() return response def pageImages(self, url, page): '''Given the HTML page that was loaded from url, grab all the images. ''' sucker = IMGSucker(url, self) sucker.feed(page) sucker.close() class WebTestCase(WebFetcher, unittest.TestCase): '''Extend the standard unittest TestCase with some HTTP fetching and response testing functions. ''' def __init__(self, methodName='runTest'): '''Initialise the server, port, authinfo, images and error_content attributes. ''' unittest.TestCase.__init__(self, methodName=methodName) WebFetcher.__init__(self) class HTTPResponse(WebFetcher, unittest.TestCase): '''Wraps a HTTP response. protocol, server, port, url - the request server and URL code, message, headers - the information returned by httplib.HTTP.getreply() body - the response body returned by httplib.HTTP.getfile() ''' def __init__(self, cookies, protocol, server, port, url, code, message, headers, body, error_content=[]): WebFetcher.__init__(self) # single cookie store per test self.cookies = cookies self.error_content = error_content[:] # this is the request that generated this response self.protocol = protocol self.server = server self.port = port self.url = url # info about the response self.code = code self.message = message self.headers = headers self.body = body self.dom = None def __str__(self): return '%s\nHTTP Response %s: %s'%(self.url, self.code, self.message) def getDOM(self): '''Get a DOM for this page ''' if self.dom is None: parser = SimpleDOMParser() try: parser.parseString(self.body) except: log('HTTPResponse.getDOM'+`(self.url, self.code, self.message, self.headers)`, self.body) raise self.dom = parser.getDOM() return self.dom def extractForm(self, path=[], include_submit=0, include_button=0): '''Extract a form (as a dictionary) from this page. The "path" is a list of 2-tuples ('element name', index) to follow to find the form. So: ..

...

...

To extract the second form, any of these could be used: [('html',0), ('body',0), ('p',1), ('form',0)] [('form',1)] [('p',1)] ''' return self.getDOM().extractElements(path, include_submit, include_button) def getForm(self, formnum, getmethod, postargs, *args): '''Given this page, extract the "formnum"th form from it, fill the form with the "postargs" and post back to the server using the "postmethod" with additional "args". NOTE: the form submission will include any "default" values from the form extracted from this page. To "remove" a value from the form, just pass a value None for the elementn and it will be removed from the form submission. example WebTestCase: page = self.get('/foo') page.getForm(0, self.post, {'name': 'blahblah', 'password': 'foo'}) or the slightly more complex: page = self.get('/foo') page.getForm(0, self.assertContent, {'name': 'blahblah', 'password': None}, 'password incorrect') ''' formData, url = self.getFormData(formnum, postargs) # whack on the url params l = [] for k, v in formData.items(): if isinstance(v, type([])): for item in v: l.append('%s=%s'%(urllib.quote(k), urllib.quote_plus(item, safe=''))) else: l.append('%s=%s'%(urllib.quote(k), urllib.quote_plus(v, safe=''))) if l: url = url + '?' + '&'.join(l) # make the post return getmethod(url, *args) def postForm(self, formnum, postmethod, postargs, *args): '''Given this page, extract the "formnum"th form from it, fill the form with the "postargs" and post back to the server using the "postmethod" with additional "args". NOTE: the form submission will include any "default" values from the form extracted from this page. To "remove" a value from the form, just pass a value None for the elementn and it will be removed from the form submission. example WebTestCase: page = self.get('/foo') page.postForm(0, self.post, {'name': 'blahblah', 'password': 'foo'}) or the slightly more complex: page = self.get('/foo') page.postForm(0, self.postAssertContent, {'name': 'blahblah', 'password': None}, 'password incorrect') ''' formData, url = self.getFormData(formnum, postargs) # make the post return postmethod(url, formData, *args) def getFormData(self, formnum, postargs={}): ''' Postargs are in the same format as the data returned by the SimpleDOM extractElements() method, and they are merged with the existing form data. ''' dom = self.getDOM() form = dom.getByName('form')[formnum] formData = form.extractElements() # Make sure all the postargs are present in the form: # TODO this test needs to be switchable, as it barfs when you explicitly # identify a submit button in the form - the existing form data doesn't # have submit buttons in it # for k in postargs.keys(): # assert formData.has_key(k), (formData, k) formData.update(postargs) for k,v in postargs.items(): if v is None: del formData[k] # transmogrify select/checkbox/radio select options from dicts # (key:'selected') to lists of values for k,v in formData.items(): if isinstance(v, type({})): l = [] for kk,vv in v.items(): if vv in ('selected', 'checked'): l.append(kk) formData[k] = l if form.hasattr('action'): url = form.action base = self.get_base_url() if not url or url == '.': if base and base[0].hasattr('href'): url = base[0].href elif self.url.endswith('/'): url = self.url elif self.url.startswith('http') or self.url.startswith('/'): url = '%s/' % '/'.join(self.url.split('/')[:-1]) else: url = '/%s/' % '/'.join(self.url.split('/')[:-1]) elif not (url.startswith('/') or url.startswith('http')): url = urlparse.urljoin(base, url) else: url = self.url return formData, url # # $Log: webunittest.py,v $ # Revision 1.13 2004/08/26 02:50:19 richard # more info # # Revision 1.12 2004/01/21 22:41:46 richard # *** empty log message *** # # Revision 1.11 2004/01/20 23:59:39 richard # *** empty log message *** # # Revision 1.10 2003/11/06 06:50:29 richard # *** empty log message *** # # Revision 1.9 2003/11/03 05:11:17 richard # *** empty log message *** # # Revision 1.5 2003/10/08 05:37:32 richard # fixes # # Revision 1.4 2003/08/23 02:01:59 richard # fixes to cookie sending # # Revision 1.3 2003/08/22 00:46:29 richard # much fixes # # Revision 1.2 2003/07/22 01:19:22 richard # patches # # Revision 1.1.1.1 2003/07/22 01:01:44 richard # # # Revision 1.11 2002/02/27 03:00:08 rjones # more tests, bugfixes # # Revision 1.10 2002/02/26 03:14:41 rjones # more tests # # Revision 1.9 2002/02/25 02:58:47 rjones # *** empty log message *** # # Revision 1.8 2002/02/22 06:24:31 rjones # Code cleanup # # Revision 1.7 2002/02/22 04:15:34 rjones # web test goodness # # Revision 1.6 2002/02/13 04:32:50 rjones # *** empty log message *** # # Revision 1.5 2002/02/13 04:24:42 rjones # *** empty log message *** # # Revision 1.4 2002/02/13 02:21:59 rjones # *** empty log message *** # # Revision 1.3 2002/02/13 01:48:23 rjones # *** empty log message *** # # Revision 1.2 2002/02/13 01:16:56 rjones # *** empty log message *** # # # vim: set filetype=python ts=4 sw=4 et si webunit-1.3.10/demo/__init__.py000777 000765 000024 00000000637 11212244064 017076 0ustar00richardstaff000000 000000 import os, unittest # figure all the modules available dir = os.path.split(__file__)[0] mods = {} l = [] __all__ = [] for file in os.listdir(dir): if not file.endswith('.py') or file == '__init__.py': continue name = file[5:-3] mods[name] = __import__(file[:-3], globals(), locals(), []) __all__.append(name) l.append(mods[name].suite()) def suite(): return unittest.TestSuite(l) webunit-1.3.10/demo/google.py000777 000765 000024 00000001271 11212244064 016606 0ustar00richardstaff000000 000000 from webunit.webunittest import WebTestCase import unittest class TestGoogle(WebTestCase): def setUp(self): self.server = 'www.google.com' def test_home(self): page = self.page('/') def test_form_good(self): page = self.page('/') page = page.getForm(0, self.page, {'q': 'Python Rules', 'btnG': 'Google Search'}) def test_form_bad(self): page = self.page('/') page = page.getForm(0, self.assertContent, {'q': 'eereerererre', 'btnG': 'Google Search'}, 'did not match any documents') def suite(): l = [ unittest.makeSuite(TestGoogle, 'test'), ] return unittest.TestSuite(l) webunit-1.3.10/demo/python_org.py000777 000765 000024 00000000523 11212244064 017521 0ustar00richardstaff000000 000000 from webunit.webunittest import WebTestCase import unittest class TestPythonOrg(WebTestCase): '''Load the front page of www.python.org ''' def test_home(self): page = self.page('http://www.python.org') def suite(): l = [ unittest.makeSuite(TestPythonOrg, 'test'), ] return unittest.TestSuite(l)