There is the following test:

@Test(timeout = 15) public void test() throws Exception { AesCrypt aes = new AesCrypt("password"); String crypted = aes.encrypt("Test string"); Assert.assertEquals("Test string", aes.decrypt(crypted)); }

In theory, it should fail if the execution time exceeds 15ms. At the same time, as seen in the screenshot, the test was performed at 23ms, but it is still considered to be passed.

Why so?

a busy cat

  • What do you want to test? Try running the test outside the IDE. - Mikhail Vaysman
  • @Mikhail lead time. Or did I misunderstand your question? - Evgeniy
  • Usually in unit tests, the timeout is used to prevent tests from hanging up. Performance unit tests do not check. What is the purpose of this test? - Mikhail Vaysman
  • @Mikhail is one of the goals - just to measure performance. But why can't JUnit be used for this? - Evgeniy
  • JUnit is not a benchmark framework. You cannot use JUnit in this capacity. Want to measure the performance of take, for example jmh - Mikhail Vaysman

2 answers 2

Using @Test(timeout = 15) not thread-safe, using it can lead to an incorrect result. Use the following code.

 public class Test { @Rule public Timeout globalTimeout= new Timeout(9); @org.junit.Test() public void test() throws InterruptedException { Thread.sleep(8); } @org.junit.Test() public void test2() throws InterruptedException { Thread.sleep(10); } }

conclusion

 org.junit.runners.model.TestTimedOutException: test timed out after 9 milliseconds at java.lang.Thread.sleep(Native Method) at Test.test2(Test.java:22) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:745)

This approach allows you to catch even 1 millisecond.

Also in jUnit5 you can use

  assertTimeoutPreemptively(ofSeconds(15), () -> { // This method takes 20 seconds to run myMethod(); } , () -> "myMethod() method took more than 15 seconds and was aborted.");

    Run the test outside the IDE and run the test before starting the test.

    • Can you please describe how to do this? - Evgeniy
    • what exactly to describe? what build system are you using? maven? gradle? - Mikhail Vaysman
    • How to run a test outside the IDE and how to do the cleanup. I do not use any collector. - Evgeniy
    • And which IDE do you use? And what type of project have you chosen? - Mikhail Vaysman
    • IntelIiJ IDEA, Kotlin project (Java) - Evgeniy