wiki:PLATFORM_STANDARDS_AUTO_TESTS_R1
Last modified 16 years ago Last modified on 01/26/09 21:33:26

Error: Macro BackLinksMenu(None) failed
compressed data is corrupt

Error: Macro TicketQuery(summary=PLATFORM_STANDARDS_AUTO_TESTS_R1, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed
current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

The aim of this revision of the task is to improve the auto testing infrastructure, conventions etc.

Task requirements

  • improve the standards for Unit Tests:
    • there are classes for which unit tests are compulsory.
    • there are classes for which unit tests are impossible and the least it can be done is integration test.
    • Note: stating the classes that should be unit tested can be done by component and team they belong to.
  • provide more information about the javadoc of unit tests:
    • should it be compulsory or not.
    • how descriptive should it be - after all these are only tests and are not supposed to be reused.
  • improve the conventions of unit tests. integration tests etc.
    • it is more suitable to provide information in the wiki instead of links to wikipedia.
  • review the helper classes and provide more if necessary.
    • make sure that assertions are enabled for unit tests.
    • provide helper classes for UI testing.
      • either provide a convenient way for testing the UI.
      • or propose a UI testing library - such libraries are already researched some time ago and these include: Abbot, !UISpec4J, JFCUnit.
        • new libraries may have appeared since then.
        • these may have been improved for the time passed.

Task result

  • wiki page.
  • source code.
  • new jars or other resources.

Implementation idea

  • review the PLATFORM_STANDARDS_AUTO_TESTS
    • add more info as stated in the requirements section.
    • review written things up to now.
  • research for libraries for GUI testing.
  • if no result from the research:
    • think of a way to test the GUI with minor source code re-factoring and more helper things.

PLATFORM_STANDARDS_AUTO_TESTS is the document where more information should be provided and unnecessary information removed. PLATFORM_STANDARDS_AUTO_TESTS_R1

How to demo

Design

The PLATFORM_STANDARDS_AUTO_TESTS page is out of date because it reflects the old testing structure of Veda. It will be recreated using the following structure: 

|_Platform Standards Auto Tests
|__General conventions
|__Unit testing
|__Integration testing
|__System testing
|__Running tests
|__GUI testing
|__Reviewing
|_Comments

Here follow the contents of each section:

  • Platform Standards Auto Tests - an overview of what the page is about.
  • General conventions - rules for writing all kind of tests and javadoc.
  • Unit/Integration/System testing - a description of the different scopes of testing - info from the currently linked wiki pages should be extracted here.
  • Running tests - a description of how to run a given test or all tests (via Maven or Hudson).
  • GUI Testing - a how-to of testing the user interface.
  • Reviewing - how to judge if a given test is good enough; how to review the testing phase of tasks.
  • Comments - Comments for improving the standards that will be incorporated into the page in the next revision.

Everyone has been asked to give suggestions for improving the standards in the Comments section. These will be considered in the implementation phase.

Research has been made on the various GUI testing libraries, starting with the following resources:
​http://asteasolutions.net/mediawiki/index.php/Sophie-JR-Research-SwingUnitTesting
​http://www.junit.org/taxonomy/term/6
It seems that Abbot (​http://abbot.sourceforge.net/doc/overview.shtml) is the best choice that suits our needs:

  • It is based on JUnit and should be easily integrated in Hudson.
  • It provides a way to add assertions for things on a modal dialog.
  • There is plenty of documentation and getting started with it should be relatively easy.

One of the other high-ranked choices, UISpec4J, is reported to have problems with the integration with Hudson and running under Linux as a whole.
A how-to of using Abbot will be provided in the implementation.

Although not stated in the task requirements, a cleanup of the existing base classes for tests will be performed (they are located in the org.sophie2.core.testing package). The reason is that there are many leftovers from the old Veda testing.

Implementation

The PLATFORM_STANDARDS_AUTO_TESTS page was thoroughly restructured and updated. Although a GUI library has been found it has some disadvantages:

  • It requires the compiled source code which does not facilitate the process of testing but instead prolongs it.
  • It requires a time-consuming first-time setup that cannot be done in this iteration due to effort consumption.

A better solution will be a plug-in that integrates with Eclipse and does not need the compiled source. Although Abbot can be used, that may happen at a later stage (for example in the next revision of this task, which is in Iteration05). Also some feedback from the release team about how the testing is going can be useful when choosing the tools to use.

TestBase has been refactored. Changeset: 1477. Old stuff like the paths and the methods for getting and creating resources have been removed. This resulted in refactoring some legacy tests: BookLegacyTest and tests found in the main.model.book.format.r1 module.

Testing

Comments

Rules should be like checklist like in other platform standards (this is related to reviewing) How-tos should be linked and added for auto tests.

  • A task requirement for the next revision should be cleaning up the base test classes - they contaion old functionality related to Veda testing. --boyan@2009-01-26