Mockk messing up with the JVM

Update: The mockk issue fas fixed on mockk 1.8.7 release, less than 10 days after I submitted the bug here.

I was working in a new service; I had been pushing commits as crazy, with their appropriate tests, and I was using kotlin, spring, jooq and mockito. Then, I had run into an issue that felt could be improved.

I had read about mockk from a coworker, so I gave it a try. I started to implement a new functionality that had nothing to do with another one and implemented the tests. The only relation they had was a table that the new functionality was reading from the database (not writing to, I promise!).

Guess what happened after that? Some other tests started to fail. Seriously, why fixing something opens several other bugs? How could be possible that a completely different test suite that had nothing to do with those services started to fail? The symptoms were really strange. I was looking at the logs,and I could see jooq retrieving a row and suddenly failing to map an object because the id field was null. As I said, that could not be possible since my DB access was readonly this time and tests were not doing anything related with db yet.

This was a really weird case, since running those tests alone worked but when invoked as part of the suite, failed. Moreover, altering the order of the tests make them work or fail, as it some dark magic was controlling the code.

Since JUnit 5 does not have yet a way to specify the order of the tests much like JUnit 4 had, this was hard to test and identify. Digging deeper I found that on the native call on java.lang.reflect.Executable, it’s private native Parameter[] getParameters0(); was returning a different set of parameters after mockk had been invoked.

Our jooq’s ParameterAwareRecordMapper indeed uses constructor.getParameters to map from a record to a POJO and voila, after that call, jooq was unable to found any parameter from the record, and thus populating a POJO with all fields initialized to null.

What jooq was doing was reading the parameter names from the constructor of the POJO and looking for fields with those names on the record. Since the POJO was generated by jooq itself based on the DB fields everything should work fine from here. It was not. This messed up my pipeline, my code and got me through the rabbit hole for at least a couple of hours.

I managed to isolate the problem in the least lines of code possible and so far I have not found any work around nor way to reset what is happening after using mockk(). I suspect that the library uses reflection to inject its own constructor but forgets to keep the parameters name. Here is the minimal test to reproduce it (remember that JUnit 5 can’t warrant test order so run them independently if needed):

There are two tests, and a data class with two parameters in the constructor, name and age. The first test shows that the class has only one constructor, but it is not the same after calling to mockk(); the second one shows that the parameter names are not present anymore.

One expects that just mocking an object does not mess with the loaded class by the JVM, but I got this:

    15:25:12.631 [main] DEBUG io.mockk.impl.instantiation.AbstractMockFactory - Creating mockk for DataClass name=#2, moreInterfaces=[]  
    15:25:12.637 [main] DEBUG io.mockk.impl.stub.CommonClearer - Clearing [] mocks
    java.lang.AssertionError: Assertion failed
    at org.tarodbofh.medium.mockk.MockkMessingNativeJVM.testConstructorEqualityAfterClearMocks(MockkMessingNativeJVM.kt:26)

This means that after calling mockk() the constructor of the class had been altered!

Of course, the second test failed:

    15:25:11.563 [main] DEBUG io.mockk.impl.instantiation.AbstractMockFactory - Creating mockk for DataClass name=#1, moreInterfaces=[]  
    15:25:12.555 [main] DEBUG io.mockk.impl.stub.CommonClearer - Clearing [] mocks
    java.lang.AssertionError: [Extracted: name]   
    Expecting:  
      <["arg0", "arg1"]>  
    to contain exactly in any order:  
      <["name", "age"]>  
    elements not found:  
      <["name", "age"]>  
    and elements not expected:  
      <["arg0", "arg1"]>
    at org.tarodbofh.medium.mockk.MockkMessingNativeJVM.testConstructorGetParametersMocked(MockkMessingNativeJVM.kt:44)

The parameter names had changed, and that is why the other tests started to fail after I mocked an object on a Unit est. Don’t forget to compile with java parameters targeting Java 8 (see https://bugs.openjdk.java.net/browse/JDK-8046108).

To do that, just add this to your .gradle script: [compileKotlin, compileTestKotlin]*.kotlinOptions*.javaParameters = true and you’ll be using the parameters. Else, the first test will fail on the first assertion and not on the second one.

If you need help to set up gradle to test this, you can have a look at my .gradle file here.

I learned this the hard way but in the end I opted to not mock any object I was requiring, but using constant value objects.

For example, if I am testing underage people I will have a val UNDERAGE_PERSON = DataClass("underage, 1) on an TestExtensions.kt file instead of using a mocked object, at least for POJOs, DTOs and other stateful constructs. For stateless and business logic methods, I am hesitant until I inspect mockk code to see what is it doing to mess with the JVM.

After a brief chat on gitter with mockk maintainer he asked to submit a bug report, so I hope this gets fixed on a future release.

Some days later, I experienced the same bug with mockito, but only when @Mock annotations where used instead of using mock(...) methods. I was unable to reproduce it on a constant behavior as apparently this happened randomly.

This bug indeed seems related to this two bugs on mockito that were already resolved.

Update: The mockk issue fas fixed on mockk 1.8.7 release, less than 10 days after I submitted the bug.

That is grade A job from the maintainers. Kudos!

Originally published in Medium