Google BDD presentation missing TestNG

I’ve been watching an excellent presentation done by Dave Astels at Google: Beyond Test Driven Development: Behavior Driven Development (hope you heard of BDD – not like in body dysmorphic disorder, but as in Dan North’s Behavior Driven Development). I know that googlers are very secretive about their work and environment, but I was kind of puzzled that nobody really mentioned TestNG when talking about better unit testing with JUnit4. Hope this is only due to the privacy policy inside Google and not because they haven’t heard of it (which would definitely be really weird).

Advertisement

TestNG revamped support for JUnit

I have spent a couple of hours this weekend on a new approach to running JUnit 3.x tests in TestNG

Previously, we have tried to emulate through TestNG the behavior of JUnit and things were quite complex. Also, considering how many extensions are there for JUnit we have never been sure we are able to run all existing JUnit tests.

So, I decided to give it another try and see if I can come out with a better approach. And instead of trying to emulate JUnit runtime behavior, I thought I can handle JUnit tests to JUnit itself and just make it report back the results. After a couple of hours of hacking I got it working and right now I think we will be able to run an even bigger percentage of existing JUnit code. And as a validation of the changes, I have run not only the TestNG tests (almost 300), but also JUnit internal tests (131) and all passed.

This new revamped support for JUnit will be released in the next version, but if you want to challenge it please feel free to build it yourself from the SVN trunk (or drop me an email) and put it to extensive work.

category: Software Testing, Testing, TestNG

TestNG gets a full book chapter

I was happy to discover that TestNG is now covered in a full chapter of the book Beginning POJOs by Brian Sam-Bodden.

The author puts aside JUnit, and introduces the reader to TestNG concepts. After a concise intro, the author builds more complex tests integrating DbUnit and EasyMock (as you may already know EasyMock 2 is already JUnit-free; in previous posts I have started posting ideas about a JUnit-free JMock).

Regarding the examples provided in the book, I have a single remark: having in mind that building a Hibernate SessionFactory is considered an expensive operation, I would use other mechanisms from TestNG that may help running the tests more quickly (by guaranteeing that the SessionFactory is created/initialized only once): @BeforeGroup (guaranteed to run once before a named group of tests), @BeforeTest (which is guaranteed to run once before the tests) or @BeforeSuite (running once before the whole suite).

category: Software Testing, Testing, TestNG, EasyMock

More on JMock and TestNG

Firstly, I would like to provide a small fix to the code provided in the previous entry: JMock for TestNG (or JUnit-free JMock)

/**
* Verify the expected behavior for the mocks registered by the current thread and
* also releases them.
*/
public static synchronized void verifyAndRelease() {
   Thread currentThread= Thread.currentThread();
   List mocks = s_mockObjects.get(currentThread);
   if(null != mocks) {
       try {
           for(Verifiable verifiable : mocks) {
               verifiable.verify();
           }
       finally {
           // make sure the mocks are released
           mocks.clear();
           s_mockObjects.put(currentThread, null);
       }
   }
}

The fix assures that even if the verification of a mock fails, the current mocks are released and will not remain around till the next test.

I have started to completely free JMock of its JUnit “addiction”. I have found three types of dependencies:

• classes in the hierarchy of org.junit.TestCase (3). These can be completely replaced by the class I have already posted
• usage of org.junit.AssertionFailureException. Having in mind that TestNG works for JVM 1.4+, we have directly used the java.lang.AssertError, and I am thinking to do the same here
• a couple of classes extending org.junit.Assert (even if they aren’t using much of it). Considering that TestNG propose a more readable form for assertions through org.testng.Assert, this change is quite trivial

I can confess that these changes are already available on my machine and I am already using them. Me and Joe have been discussing about the possibility to integrate these changes directly in JMock. If this will not happen, than I will most probably include the code in TestNG.

A last point (one that I just get me burnt tonite), is that JMock doesn’t check (or I haven’t figured how to make it) the order of expectations, so from its perspective the following code is same correct (or wrong):

   [...]
   mock.expect(once()).method("methodOne");
   mock.expect(once()).methog("methodTwo");
   [...]
   mock.verify();

and

   [...]
   mock.expect(once()).method("methodTwo");
   mock.expect(once()).methog("methodOne");
   [...]
   mock.verify();

will both pass or fail disregarding the real order of invocation. I agree that in most of the cases, JMock behavior is enough good, but not having a way to force the order of invocation is in my opinion a missing feature. I am gonna try to add this feature to my code.

category: Software Testing, Testing, TestNG, JMock, EasyMock, JUnit

JMock for TestNG (or JUnit-free JMock)

Last days I have created a small utility class that would allow me to work with JMock without depending on JUnit. If you develop tests using TestNG and you need mocks a la JMock, than here it is (I have removed the imports so that the listing doesn’t go too long):

/**
 * An utility class to allow static usage of JMock (removes the restriction
 * to subclass JMock specific classes).
 * 
 * @author Alexandru Popescu
 */
public class JMock {
    static final Constraint ANYTHING = new IsAnything();
    private static final WeakHashMap<Thread, List<Verifiable>> s_mockObjects = 
            new WeakHashMap<Thread, List<Verifiable>>();
    
    /**
     * Creates a mock object that mocks the given type.  
     * The mock object is named after the type;  the exact
     * name is calculated by {@link #defaultMockNameForType}.
     *
     * @param mockedType The type to be mocked.
     * @return A {@link Mock} object that mocks <var>mockedType</var>.
     */
    public static Mock mock(Class mockedType) {
        return mock(mockedType, defaultMockNameForType(mockedType));
    }

    /**
     * Creates a mock object that mocks the given type and is explicitly given a name.
     * The mock object is named after the type;  
     * the exact name is calculated by {@link #defaultMockNameForType}.
     *
     * @param mockedType The type to be mocked.
     * @param roleName The name of the mock object
     * @return A {@link Mock} object that mocks <var>mockedType</var>.
     */
    public static Mock mock( Class mockedType, String roleName ) {
        Mock newMock = new Mock(newCoreMock(mockedType, roleName));
        registerToVerify(newMock);
        return newMock;
    }
    
    public static Mock mock(Class mockedClass, 
                            String roleName, 
                            Class[] constructorArgumentTypes, 
                            Object[] constructorArguments) {
        Mock newMock = new Mock(newClassCoreMock(mockedClass, 
                                                 roleName, 
                                                 constructorArgumentTypes, 
                                                 constructorArguments));
        registerToVerify(newMock);
        return newMock;
    }

    public static Mock mock(Class mockedClass, 
                            Class[] constructorArgumentTypes, 
                            Object[] constructorArguments) {
        return mock(mockedClass, 
                    defaultMockNameForType(mockedClass), 
                    constructorArgumentTypes, 
                    constructorArguments);
    }

    public static Stub returnValue(Object o) {
        return new ReturnStub(o);
    }

    public static Stub returnValue(boolean result ) {
        return returnValue(new Boolean(result));
    }

    public static Stub returnValue(byte result ) {
        return returnValue(new Byte(result));
    }

    public static Stub returnValue(char result ) {
        return returnValue(new Character(result));
    }

    public static Stub returnValue(short result ) {
        return returnValue(new Short(result));
    }

    public static Stub returnValue(int result ) {
        return returnValue(new Integer(result));
    }

    public static Stub returnValue(long result ) {
        return returnValue(new Long(result));
    }

    public static Stub returnValue(float result ) {
        return returnValue(new Float(result));
    }

    public static Stub returnValue(double result ) {
        return returnValue(new Double(result));
    }

    public static Stub returnIterator(Collection collection) {
        return new ReturnIteratorStub(collection);
    }
    
    public static Stub returnIterator(Object[] array) {
        return new ReturnIteratorStub(array);
    }
    
    public static Stub throwException(Throwable throwable) {
        return new ThrowStub(throwable);
    }

    public static InvocationMatcher once() {
        return new InvokeOnceMatcher();
    }

    public static InvocationMatcher atLeastOnce() {
        return new InvokeAtLeastOnceMatcher();
    }

    public static InvocationMatcher exactly(int expectedCount) {
        return new InvokeCountMatcher(expectedCount);
    }
    
    public static InvocationMatcher never() {
        return new TestFailureMatcher("not expected");
    }

    public static InvocationMatcher never( String errorMessage ) {
        return new TestFailureMatcher("not expected ("+errorMessage+")");
    }
    
    public static Stub onConsecutiveCalls(Stub... stubs) {
        return new StubSequence(stubs);
    }
    
    public static Stub doAll(Stub... stubs) {
        return new DoAllStub(stubs);
    }
    
    public static IsEqual eq(Object operand) {
        return new IsEqual(operand);
    }

    public static IsEqual eq(boolean operand ) {
        return eq(new Boolean(operand));
    }

    public static IsEqual eq(byte operand ) {
        return eq(new Byte(operand));
    }

    public static IsEqual eq(short operand ) {
        return eq(new Short(operand));
    }

    public static IsEqual eq(char operand ) {
        return eq(new Character(operand));
    }

    public static IsEqual eq(int operand ) {
        return eq(new Integer(operand));
    }

    public static IsEqual eq(long operand ) {
        return eq(new Long(operand));
    }

    public static IsEqual eq(float operand ) {
        return eq(new Float(operand));
    }

    public static IsEqual eq(double operand ) {
        return eq(new Double(operand));
    }

    public static IsCloseTo eq( double operand, double error ) {
        return new IsCloseTo(operand, error);
    }

    public static IsSame same( Object operand ) {
        return new IsSame(operand);
    }

    public static IsInstanceOf isA( Class operandClass ) {
        return new IsInstanceOf(operandClass);
    }

    public static StringContains stringContains(String substring) {
        return new StringContains(substring);
    }

    public static StringContains contains(String substring) {
        return stringContains(substring);
    }
    
    public static StringStartsWith startsWith(String substring ) {
        return new StringStartsWith(substring);
    }
    
    public static StringEndsWith endsWith(String substring ) {
        return new StringEndsWith(substring);
    }
    
    public static IsNot not(Constraint c ) {
        return new IsNot(c);
    }

    public static And and(Constraint left, Constraint right ) {
        return new And(left, right);
    }

    public static Or or(Constraint left, Constraint right ) {
        return new Or(left, right);
    }

    /// HINT: not sure about these following 3
    public static Object newDummy( Class dummyType ) {
        return Dummy.newDummy(dummyType);
    }

    /// HINT: what is this?
    public static Object newDummy( Class dummyType, String name ) {
        return Dummy.newDummy(dummyType, name);
    }

    /// HINT: what is this?
    public static Object newDummy( String name ) {
        return Dummy.newDummy(name);
    }
    
    public static void assertThat(Object actual, Constraint constraint) {
        if (!constraint.eval(actual)) {
            StringBuffer message = new StringBuffer("\nExpected: ");
            constraint.describeTo(message);
            message.append("\n    got : ").append(actual).append('\n');
            throw new AssertionError(message);
        }
    }

    public static void assertThat(boolean actual, Constraint constraint) {
        assertThat(new Boolean(actual), constraint);
    }

    public static void assertThat(byte actual, Constraint constraint) {
        assertThat(new Byte(actual), constraint);
    }

    public static void assertThat(short actual, Constraint constraint) {
        assertThat(new Short(actual), constraint);
    }

    public static void assertThat(char actual, Constraint constraint) {
        assertThat(new Character(actual), constraint);
    }

    public static void assertThat(int actual, Constraint constraint) {
        assertThat(new Integer(actual), constraint);
    }

    public static void assertThat(long actual, Constraint constraint) {
        assertThat(new Long(actual), constraint);
    }

    public static void assertThat(float actual, Constraint constraint) {
        assertThat(new Float(actual), constraint);
    }

    public static void assertThat(double actual, Constraint constraint) {
        assertThat(new Double(actual), constraint);
    }
    
    public static HasPropertyWithValue hasProperty(String propertyName, Constraint expectation) {
        return new HasPropertyWithValue(propertyName, expectation);
    }
    
    public static HasProperty hasProperty(String propertyName) {
       return new HasProperty(propertyName);
    }
    
    public static HasToString toString(Constraint toStringConstraint) {
        return new HasToString(toStringConstraint);
    }
    
    public static IsCompatibleType compatibleType(Class baseType) {
        return new IsCompatibleType(baseType);
    }
    
    public static IsIn isIn(Collection collection) {
        return new IsIn(collection);
    }
    
    public static IsIn isIn(Object[] array) {
        return new IsIn(array);
    }
    
    public static IsCollectionContaining collectionContaining(Constraint elementConstraint) {
        return new IsCollectionContaining(elementConstraint);
    }
    
    public static IsCollectionContaining collectionContaining(Object element) {
        return collectionContaining(eq(element));
    }
    
    public static IsArrayContaining arrayContaining(Constraint elementConstraint) {
        return new IsArrayContaining(elementConstraint);
    }

    public static IsArrayContaining arrayContaining(Object element) {
        return arrayContaining(eq(element));
    }

    public static IsArrayContaining arrayContaining(boolean element) {
        return arrayContaining(new Boolean(element));
    }

    public static IsArrayContaining arrayContaining(byte element) {
        return arrayContaining(new Byte(element));
    }

    public static IsArrayContaining arrayContaining(short element) {
        return arrayContaining(new Short(element));
    }

    public static IsArrayContaining arrayContaining(char element) {
        return arrayContaining(new Character(element));
    }

    public static IsArrayContaining arrayContaining(int element) {
        return arrayContaining(new Integer(element));
    }

    public static IsArrayContaining arrayContaining(long element) {
        return arrayContaining(new Long(element));
    }

    public static IsArrayContaining arrayContaining(float element) {
        return arrayContaining(new Float(element));
    }

    public static IsArrayContaining arrayContaining(double element) {
        return arrayContaining(new Double(element));
    }
    
    public static IsMapContaining mapContaining(Constraint keyConstraint, Constraint valueConstraint) {
        return new IsMapContaining(keyConstraint, valueConstraint);
    }
    
    public static IsMapContaining mapContaining(Object key, Object value) {
        return mapContaining(eq(key), eq(value));
    }

    public static IsMapContaining mapWithKey(Object key) {
        return mapWithKey(eq(key));
    }

    public static IsMapContaining mapWithKey(Constraint keyConstraint) {
        return new IsMapContaining(keyConstraint, ANYTHING);
    }

    public static IsMapContaining mapWithValue(Object value) {
        return mapWithValue(eq(value));
    }

    public static IsMapContaining mapWithValue(Constraint valueConstraint) {
        return new IsMapContaining(ANYTHING, valueConstraint);
    }

    /**
     * Verify the expected behavior for the mock registered by the current thread.
     */
    public static void verify() {
        List<Verifiable> mocks = s_mockObjects.get(Thread.currentThread());
        if(null != mocks) {
            for(Verifiable verifiable : mocks) {
                verifiable.verify();
            }
        }
    }
    
    /**
     * Verify the expected behavior for the mocks defined as fields of the arguement object.
     * 
     * @param object the object to be inspected
     */
    public static void verifyObject(Object object) {
        Verifier.verifyObject(object);
    }

    /**
     * Helper method that delegates to {@link #verify()} and {@link verifyObject(Object)}.
     */
    public static void verifyAll(Object object) {
        verify();
        verifyObject(object);
    }
    
    /**
     * Verify the expected behavior for the mocks registered by the current thread and
     * also releases them.
     */
    public static synchronized void verifyAndRelease() {
        Thread currentThread= Thread.currentThread();
        List<Verifiable> mocks = s_mockObjects.get(currentThread);
        if(null != mocks) {
            for(Verifiable verifiable : mocks) {
                verifiable.verify();
            }
        }
        mocks.clear();
        s_mockObjects.put(currentThread, null);
    }
    
    /**
     * Helper method delegating to {@link #verifyAndRelease()} and {@link #verifyObject(Object)}.
     */
    public static void verifyAllAndRelese(Object object) {
        verifyAndRelease();
        verifyObject(object);
    }
    
    ///
    private static synchronized void registerToVerify(Verifiable verifiable) {
        List<Verifiable> mocks = s_mockObjects.get(Thread.currentThread());
        if(null == mocks) {
            mocks = new ArrayList<Verifiable>();
            s_mockObjects.put(Thread.currentThread(), mocks);
        }
        
        mocks.add(verifiable);
    }

    private static DynamicMock newCoreMock(Class mockedType, String roleName ) {
        return new CoreMock(mockedType, roleName);
    }

    private static DynamicMock newClassCoreMock(Class mockedClass, 
                                                String roleName,
                                                Class[] constructorArgumentTypes, 
                                                Object[] constructorArguments) {
        return new CGLIBCoreMock(mockedClass, roleName, constructorArgumentTypes, constructorArguments);
    }

    /**
     * Calculates a default role name for a mocked type.
     * @param mockedType
     * @return
     */
    private static String defaultMockNameForType(Class mockedType) {
        return "mock" + Formatting.classShortName(mockedType);
    }
}

It works for mocking both interfaces and classes (through CGLIB).
And here is a short example of usage (JDK5, because I use static imports):

public class TypeAccessFilterTest {
    @Test
    public void jmockBotNormalRequest() throws IOException, ServletException {
        // static imports
        Mock mockBotChecker = mock(BotChecker.class, new Class[0], new Object[0]);
        Mock mockFilterChain= mock(FilterChain.class);
        TypeAccessFilter taf= new TypeAccessFilter();
        taf.setBotChecker((BotChecker) mockBotChecker.proxy());
        
        mockFilterChain.expects(once()).method("doFilter")
                       .with(isA(HttpServletRequest.class),
                             isA(JSessionidBeautifierHttpResponseWrapper.class));
        
        taf.doFilter(m_mockRequest, m_mockResponse, (FilterChain) mockFilterChain.proxy());
        
        verifyAndRelease();
    }

As far as I know the latest versions of EasyMock allow the same JUnit-free usage, which in my opinion is a great thing.

category: Software Testing, Testing, TestNG, JMock, EasyMock, JUnit

TestNG 5.0 – new annotations and more

The new version of TestNG was released. Cedric made the first announcement and the news spreaded quite fast.
The main addition in the new version are the completely revamped annotations. The old @Configuration was replaced by more clear and much more readable annotations: @Before/AfterSuite, @Before/AfterTest, @Before/AfterGroup, @Before/AfterClass, @Before/AfterMethod (phewww… quite a few). But check the following code to see how much better it is:

The “old” way:

public class MethodCallOrderTest {
  @Configuration(beforeTestClass=true)
  public void beforeClass() {
     // do before any test methods in this class are executed
  }
  
  @Configuration(beforeTestMethod=true)
  public void beforeMethod() {
     // do before each test method
  }
  
  @Test
  public void realTest() {
  }
  
  @Configuration(afterTestMethod=true)
  public void afterMethod() {
     // do after each test method
  }
  
  @Configuration(afterTestClass=true)
  public void afterClass() {
  }

  @Configuration(afterSuite=true)
  public void cleanUp() {
      // do once after all tests
  }

}

new way:

public class MethodCallOrderTest {
  @BeforeClass
  public void beforeClass() {
     // hey it looks like I don't need to put any comment here to explain it, ain't it?
  }

  @BeforeMethod
  public void beforeMethod() {
  }
  
  @Test
  public void realTest() {
  }
  
  @AfterMethod
  public void afterMethod() {
  }
  
  @AfterClass
  public void afterClass() {
  }
  
  @AfterSuite
  public void cleanUp() {
  }
}

There are a lot more improvements in this new version: better reports, filtered stack traces, more configuration options for the Ant task.
Last, but not least: don’t worry about the next versions. We already have in mind a couple of more nice things that will make life easier and a lot more pleasant for test developers.

category: Software Testing, Testing, TestNG

JUnit release and state of TestNG

Two weeks ago JUnit has announced the new version. I was waiting for this release for quite a long time… but not because of the features of JUnit 4, because most of them (or all) have been available in TestNG a long time before. In a previous post I have done some comparisons of JUnit 4 features with TestNG features (and Cedric did it too: JUnit4 Overview). So, my waiting was not about the feature list, but rather to see the reactions of people in the testing space. And it looks like my expectations were meet. I just have a few links about reactions, and here there are:

• Geert Bevin
• Tim Fennel and some more on the TSS thread
• Bill Dudney

More interesting posts about TestNG can be found here.

Enjoy your tests with TestNG!

category: JUnit, TestNG, testing

JUnit 4.0

Read post JUnit 4.0.

TestNG plugin for IDEA – help needed

Hi!

I would like to know if somebody out there would be interested in starting a TestNG plugin for
IntelliJ IDEA. My knowledge of Idea API is almost 0 (and their plugin development distro left me at
exactly the same level), but I would gladly help to have this task done.

If anybody is interested in doing this, please drop me an email to plan it out.

Thanks in advance and I really hope that somebody will help me out.

HTMLUnit and DWR

The last nite I was trying to write down some unit/integration tests using htmlunit embedded in TestNG.
While for simple pages this was quite neat, I have a page using DWR for filling some select elements (triggered by page onload and then onchange events). Here i faced a set of problems to which unfortunately i wasn’t able to find a solution:

• the htmlunit XMLHttpRequest doesn’t solve the src attribute and after this a NPE occurs in jsxFunction_open
  I have patched this by making HtmlScript.getSrcAttribute solve the complete URL
• the 2nd problem was that again htmlunit could not solve the dynamic generated call of DWR: DWREngine._execute […]
  I’ve been able to patch this too by making the DWRServlet use the complete URL for the dynamic call
• the 3rd and final problem – the unsolved one was that htmlunit XMLHttpRequest.jsxFunction_send is expecting as a result a text/xml that must be an XmlPage

I would really like to have this test working so if any of the readers can enlighten me how to fix this or I have the luck that one of the developers will read this, it would be really really great.