Lain and Sakura

Icono

”One byte to rule them all’ – dtmf – old school

Test, tests, and more tests

Unit Testing

Obtenido de: http://plone.org/documentation/tutorial/five-zope3-walkthrough/tests

Author: Jean-Francois Roche http://plone.org/author/jfroche

Test driven development has proved to be a really great practice for productive developers. And that’s why writing tests for every new functionality, for every change and bug fix, is now done by most of the Zope/Plone developer community. (So much so that if you submit code and want your code released, don’t imagine it will see the light of day without tests!).

Writing tests takes time, and you must take the time to write them. Once written that time will be payed back many times. Well planned tests will show you that your changes, bug fixes, and refactoring didn’t create other bugs in your code, and will prevent you losing time looking for them.

Testing is magic, it transforms a developer into a user. When you are writing your tests you place yourself in the user’s skin and begin to see how he might use your code. This can show you how nice your code is: where are the obscure parts, the hard to understand methods, the wrong class decompositions, etc.

You know that one of the biggest assets of Plone is its community. Tests are even more important in collective code for two main reasons:

  1. You want to share with others your functionalities, your way of coding, your way of thinking and show the right way how to use these functionalities.
  2. Many different people can/could write inside your code, if many tests are already written they can assure themselves that the code they write doesn’t break yours.
  3. It covers you when the maintainer, or release manager come to scream at you that you broke something with your last changes.

So all this is about verifying your code and explaining it to others.

Tests should:

  • Be repeatable
  • Run without human intervention needed
  • Be concise
  • Tell a story
  • Not test obvious things
  • Be deterministic

PloneTestCase

When you create tests for Plone, you don’t want to lose time installing a Plone portal, or other basic Zope/Plone products. So to get quicker test results, we use a unit test framework that create automated unit tests suites. Plone has its own: PloneTestCase. As Plone is based on Zope, PloneTestCase is a layer on the top of the ZopeTestCase (the zope unit test framework – which is based on Python’s unittest package [and if you want to know the whole story, python’s unittest package is based on Java’s JUnit and the Smalltalk testing framework]). This framework is a huge help for running your test quickly, often and with clear results.

Vocabulary

By test we mean a test method.

By unittest we mean a class which contains all the test methods (if you want to use the Plone test framework, this class should inherit from PloneTestCase).

By unittest suite we mean a collection of unittest.

We will describe here a bunch of basic things available in PloneTestCase and that we use a lot in the next sections.

Products Installed:

Here is the list of default installed products. If you need additional products you will have to install them explicitly with the method we describe later.

Zope

  • ZCTextIndex
  • MailHost
  • PageTemplates
  • PythonScripts
  • ExternalMethod
  • GroupUserFolder
  • Five

CMF

  • CMFCore
  • CMFDefault
  • CMFCalendar
  • CMFTopic
  • DCWorkflow
  • CMFUid
  • CMFActionIcons
  • CMFQuickInstallerTool
  • CMFFormController

Plone – Archetypes

  • Archetypes
  • MimetypesRegistry
  • PortalTransfroms
  • ATContentTypes
  • ATReferenceBrowserWidget
  • CMFDynamicViewFTI
  • ExternalEditor
  • ExtendedPathIndex
  • ResourceRegistries
  • SecureMailHost
  • kupu

and, last but not least, CMFPlone.

In Plone 2.5 other important products are also installed: CMFPlacefulWorkflow, PlonePAS…

Objects installed:

Here are the objects you can use when you instantiate a PloneTestCase:

  • self.portal : a fresh Plone Portal install with all the portal tools you need inside.
  • self.folder : when running a PloneTestCase you are logged as a default user. This (empty) folder is the home folder of the default user. As it will be important for you to be able to do everything you need in this folder, default user is the owner of this folder.

Useful Methods you can use:

Here are the methods you can use on the self object (inside the PloneTestCase instance):

  • addProduct(name) : Uses the quickinstaller to install a products inside the Plone portal (self.portal). So if you defined your Product and your content type, don’t forget to install it in the Plone Portal before trying to invoke it.
  • setRoles(roles, name=default_user) : Change the current user’s roles (roles can be a string, a tuple or a list). Really important if you want to check security issues. You can also change the roles of other users by setting the name parameter.
  • setGroup(groups, name=default_user) : Change the current user’s groups (groups can be a string, a tuple or a list). You can also change the groups for other users by setting the name parameter.
  • setPermissions(permissions, role) : Change the permissions on the portal object for the role. Permissions can be a string, a tuple or a list. Role must be a string.
  • login(name) : It’s sometimes clearer to create new users with different roles or groups and after that login in as these users.
  • logout() : You want to be relegated to Anonymous inside the Plone instance? Use this method.

Unit Test Setup

The testing framework will run all the methods inside any class where the method name starts with test. So testMethod1(self) will be automatically run by the framework and you won’t have to bother anymore about explicitly calling it somewhere.

You might often want to repeat the same initialization before calling your test method. The framework gives you an powerful method for that:

  • afterSetUp(self) : You should put in this method all the code you want to do before running each of your test methods.

Unit Test Setup

Assertion Testing Methods (python unittest based)

With test there is a known input and an expected output. This input-output correctness is checked by assertion. Python unittest package give us a range of methods to test assertions:

  • failIf(expression) : Fail the test if the expression is true.
  • failUnless(expression) : Fail the test unless the expression is true.
  • failUnlessEqual(first, second) : Fail if the two objects are unequal as determined by the == operator.
  • failIfEqual(first, second) : Fail if the two objects are equal as determined by the == operator.
  • fail(msg) : Fail immediately, with the given message.
  • failUnlessRaises(excClass, callableObj, args, *kwargs) : Fail unless an exception of class excClass is thrown by callableObj when invoked with arguments args and keyword arguments kwargs.

Failure and Errors

Failures and Errors are two different things!

Failures occur when an assertion has failed (you were expecting the opposite result from the test assertion).

Errors occur when something you didn’t expect occurs (exceptions, errors in your code…).

Let’s create a test! You learn better with practice. You will see it’s easy.

Common practice is to create a test class for each class you want to test and one (and sometimes more) test method for each important method in your class (getter and setter are often left untested due to their obviousness).

First download the PloneTestCase (http://plone.org/products/plonetestcase or from svn https://svn.plone.org/svn/collective/PloneTestCase/trunk/) and extract it in your favourite Zope Products folder (let’s assume that you installed Plone in there ;)).

Let’s take the PloneTestCase class with all the things we need inside:

    >>> from Products.PloneTestCase import PloneTestCase

Let’s say we want to test some of the Plone Document (ATDocument) behaviour. Let’s create a class which will use this great PloneTestCase we have just imported:

    class TestATDocument(PloneTestCase):
         """
           A basic test case for Plone Document
         """
         pass

Here it is we have done our first Plone test case. Not hard ? Yes i agree this doesn’t test much :). Let’s test two things:

1) when I edit the title of my document, I want it to be edited correctly (I agree that we are basically testing obvious thing here, let’s keep things simple).

2) when I add a document, I want to it to be inside the Plone Catalog.

As you see in these two tests we will need a basic document created, so let’s do it once in the afterSetUp method so that our document will be created before each test.

Remember that each testing method must begin with test. Let’s create testDocument.py:

     class TestATDocument(PloneTestCase):
          """
            A less basic test case for Plone Document
          """
          def afterSetUp(self):
             """
               Let's create in our home folder the document we need
             """
             self.folder.invokeFactory('Document', id='doc')
             # We now have a document with id "doc" inside our home folder

          def testEditTitle(self):
             """
              Let's see if a title change on the document goes well
             """
             self.folder.doc.setTitle('A wonderful document title')
             self.assertEqual(self.folder.doc.Title, 'A wonderful document title')
             # this will fail if the setTitle didn't  correctly do its job!

          def testDocumentInCatalog(self):
             """
               Let's see if the document is in the catalog
             """
             # the catalog is in the Plone portal
             self.failUnless(self.portal.portal_catalog(getId='doc'))

And there it is. If this passes we can be sure that we can change the title of a document and that once created a document is in the plone catalog.

Now comes the time to include our fresh testcase inside a testsuite and to run our tests.

To be able to run this you will need two files :

  • framework.py : To be able to run test from python you will need to setup a few PATHs, this file will do most of the job for you.
  • runalltests.py : This small python code will just run all the files in the current directory which begin the the word test.

Copy these files from the PloneTestCase folder to the folder where all your test cases are (often the “tests” folder).

So, to run the test suite, we will need to decorate our PloneTestCase. To add a bit of difficulty I want also to install a product inside my portal which isn’t provided in the above list. Let’s say I want to use the Plone Language Tool (I agree, we won’t need it for executing our test):

       # First, above all, execute the framework.py

       import os, sys 
       if __name__ == '__main__':
          execfile(os.path.join(sys.path[0], 'framework.py'))

       # Install the PloneLanguageTool Product in Zope

       from Testing import ZopeTestCase
       ZopeTestCase.installProduct('PloneLanguageTool')

       # Initialize our Plone and default install PloneLanguageTool in it

       from Products.PloneTestCase import PloneTestCase
       PloneTestCase.setupPloneSite(products=['PloneLanguageTool'])

       # Here it is, everything installed. We can put here our testcase...

       class TestATDocument(PloneTestCase.PloneTestCase):
             """
               A less basic test case for Plone Document
             """
             def afterSetUp(self):
                """
                  Let's create in our home folder the document we need
                """
                self.folder.invokeFactory('Document', id='doc')
                # We now have a document with id "doc" inside our home folder

             def testEditTitle(self):
                """
                  Let's see if a title change on the document goes well
                """
                self.folder.doc.setTitle('A wonderful document title')
                self.assertEqual(self.folder.doc.Title(), 'A wonderful document title')
                # this will fail if the setTitle didn't correctly do its job!

             def testDocumentInCatalog(self):
                """
                  Let's see if the document is in the catalog
                """
                # the catalog is in the Plone portal
                self.failUnless(self.portal.portal_catalog(getId='doc'))

       # Now we need our testcase inside a test suite.

       def test_suite():
            from unittest import TestSuite, makeSuite
            suite = TestSuite()
            suite.addTest(makeSuite(TestATDocument))
            return suite

       # and if you want to be able to run your suite directly (python testDocument.py)

       if __name__ == '__main__':
            framework()

Everything is set up now. Last thing to do is to say where your zope is in your system. On Unix based system you can do this like so :

      export SOFTWARE_HOME=/usr/lib/zope2.9/lib/python

Now you have two ways to run your test suite, either

  • “python runalltests.py” : Which will look in every file with the name beginning with “test” and run all defined test suites.
  • “python testDocument.py” : Which will run the specified test suite.

While running you will see

1) Installation of the Zope – Products

2) Once executed, a single test (method) will be represented by:

“.” : which means that your test ran correctly.

“F” : which means that your test failed (you will get more information at the end).

“E” : which means that your test has error (you will get more information at the end).

Each time a test fails or has an error you will get a traceback and more verbose information about the failure/error.

DocTest

What do you see in the word “DocTest”? Doc and Test. So a doctest is documentation and, at the same time, a test that proves that your code is working.

Many argue that people should read unit tests and they should be clear enough so that no more verbose comments should be added. It’s correct that tests should be clear but I wouldn’t be that strict. I think the more people I can explain my code to, the more feedback I will get .

Although we consider test cases as developer documentation, doctest is considered as a middle technique between documentation and test case. No more stale and useless documentation! Doctest enables living documentation, always in step with the current implementation.

A doctest is a text, or structured text file (which should be written inside the docs folder of your package/products). So inside this file you will explain your code and at the same time you will be able to call python code. To call python code just do:

    >>>

This represents a call to the python interpreter. Around it you can place your explanation. If your python code returns something, you have to do exactly the same as if you would call this code from a python interpreter session. For example:

    >>> print 'hello world'
    hello world

The return value must be written at the same indentation level as the >>>

One problem is that one doctest represents in itself more than one test. You want to show multiple things inside your doctest, but afterSetUp is only run once before execution of the whole doctest. One doctest represents one python session. So if i do:

    >>> a = 'hello'

My variable a will be set to hello until the end of the document. Never forget that, it could lead to some big problems!

By the way, all this document is a doctest for the ATContentTypes products. It can be executed there.

Vocabulary:

  • a doctest “file” will represent the txt file which include our doctest.
  • a doctest “class” will represent the unittest class that defines a doctest.

Once written, the doctest file should be linked to a testcase class and a testsuite. So let’s see how do we setup a doctest in the test part (this should go inside a python file in the tests folder – with a file name which begins with test):

      # Like before we use the framework.py

      import os, sys
      if __name__ == '__main__':
           execfile(os.path.join(sys.path[0], 'framework.py'))

      # We install plone as usual. We want to test plone related stuff in
      # our doctest

      from Products.PloneTestCase import PloneTestCase
      PloneTestCase.setupPloneSite()

      # then we need the zope doctestsuite and link our doctest text file
      # with a functional test case

      from Testing.ZopeTestCase import FunctionalDocFileSuite
      from Products.PloneTestCase.PloneTestCase import FunctionalTestCase

      # we have a doctest file named archive.txt which is located in
      # ATContentTypes inside the docs folder (I say it again, the doctest file
      # should always be inside the docs folder, not in tests folder).

      def test_suite():
            import unittest
            suite = unittest.TestSuite()
            suite.addTest(FunctionalDocFileSuite('archive.txt',
                                                 package="Products.ATContentTTypes.docs",
                                                 test_class=FunctionalTestCase
                                                 )
                            )
      if __name__ == '__main__':
            framework()

As usual you can run this file directly, or just run python runalltests.py.

Now imagine that you want to prepare some things inside your testcase class before running your doctest file (archive.txt). It’s easy, just create your test class which inherits from FunctionalTestCase, define the afterSetUp method and change the test suite to use your class. Let’s do it…

Be careful with this, it could confuse the people reading your doctest file if you don’t explain clearly that you have already created tests in the doctest class.

We keep it basic. We imagine that we really need to create a document inside the home folder but don’t need to show that in the doctest file:

      import os, sys
      if __name__ == '__main__':
           execfile(os.path.join(sys.path[0], 'framework.py'))

      from Products.PloneTestCase import PloneTestCase
      PloneTestCase.setupPloneSite()

      # we will now subclass FunctionalTestCase and define our afterSetUp method
      from Products.PloneTestCase.PloneTestCase import FunctionalTestCase

      class TestArchiveWithDocument(FunctionalTestCase):
            """
               Our Functional test class with a document inside
            """

            def afterSetUp(self):
                """
                   Creating a document in the home directory that the archive doc test can use
                """
                self.folder.invokeFactory('Document', id='doc')

      # And that's all! No test method, the only test method will be our
      # doctest file.  

      # now we need to link our functional test class to our doctest file
      # inside a test suite:
      from Testing.ZopeTestCase import FunctionalDocFileSuite
      def test_suite():
            import unittest
            suite = unittest.TestSuite()
            suite.addTest(FunctionalDocFileSuite('archive.txt',
                                                 package="Products.ATContentTTypes.docs",
                                                 test_class=TestArchiveWithDocument
                                                 )
                            )
      if __name__ == '__main__':
            framework()

Now you know everything about tests, you have no excuse anymore for not writing them!

Anuncios

Archivado en: Plone, Python

Responder

Introduce tus datos o haz clic en un icono para iniciar sesión:

Logo de WordPress.com

Estás comentando usando tu cuenta de WordPress.com. Cerrar sesión / Cambiar )

Imagen de Twitter

Estás comentando usando tu cuenta de Twitter. Cerrar sesión / Cambiar )

Foto de Facebook

Estás comentando usando tu cuenta de Facebook. Cerrar sesión / Cambiar )

Google+ photo

Estás comentando usando tu cuenta de Google+. Cerrar sesión / Cambiar )

Conectando a %s

A %d blogueros les gusta esto: