Why is maven surefire running my tests twice in a single execution since switching on testng parallel threads?

[Raluca Morariu]

Hi guys,

I have recently tried to improve the time for test execution by setting up
parallel threads for them. I have tried to it in two ways:
1. In my testsuite .xml file like so

<suite name="test" verbose="1" parralel="methods" thread-count="30">
<test name="RegressionTests" preserve-order="false">

2. in my pom.xml

<configuration>
<suiteXmlFiles>
<suiteXmlFile>${testSuite}</suiteXmlFile>
</suiteXmlFiles>
<parallel>methods</parallel>
<threadCount>30</threadCount>
</configuration>

When running the tests in Eclipse, they seem to be running in paralle
because the time for the execution improves. But when running the tests
using maven with the surefire plugin it seems that maven runs the tests
twice which is not much improvement when dealing with a lot of tests. This
is my maven output. I require maven in order to run tests in CD with
Jenkins.

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
log4j:WARN No appenders could be found for logger
(org.springframework.test.context.support.DependencyInjectionTestExecutionListener).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.174 sec

Results :

Tests run: 76, Failures: 0, Errors: 0, Skipped: 0

[INFO] [jar:jar {execution: default-jar}]
[INFO] Building jar:
D:\Perforce\depot\se\development\services\customer-readonly-service\HEAD\Test\target\customer-readonly-service-test-1.1-SNAPSHOT.jar
[INFO] Preparing cobertura:check
[WARNING] Removing: check from forked lifecycle, to prevent recursive
invocation.
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] Copying 1 resource
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [cobertura:instrument {execution: default}]
[WARNING] No files to instrument.
[INFO] NOT adding cobertura ser file to attached artifacts list.
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] Copying 18 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory:
D:\Perforce\depot\se\development\services\customer-readonly-service\HEAD\Test\target\surefire-reports

-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.

Tests run: 76, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.011 sec

Results :

Tests run: 76, Failures: 0, Errors: 0, Skipped: 0

------------------------------
Many thanks for the help

Raluca

[Abraham Lin]

It looks like something in your Maven configuration is incorrect. What does
the rest of the POM look like, and how are you invoking the build process?

Post by Raluca Morariu
Hi guys,
I have recently tried to improve the time for test execution by setting up
1. In my testsuite .xml file like so
<suite name="test" verbose="1" parralel="methods" thread-count="30">
<test name="RegressionTests" preserve-order="false">
2. in my pom.xml
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${testSuite}</suiteXmlFile>
</suiteXmlFiles>
<parallel>methods</parallel>
<threadCount>30</threadCount>
</configuration>
When running the tests in Eclipse, they seem to be running in paralle
because the time for the execution improves. But when running the tests
using maven with the surefire plugin it seems that maven runs the tests
twice which is not much improvement when dealing with a lot of tests. This
is my maven output. I require maven in order to run tests in CD with
Jenkins.
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
log4j:WARN No appenders could be found for logger
(org.springframework.test.context.support.DependencyInjectionTestExecutionListener).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.174 sec
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0
[INFO] [jar:jar {execution: default-jar}]
D:\Perforce\depot\se\development\services\customer-readonly-service\HEAD\Test\target\customer-readonly-service-test-1.1-SNAPSHOT.jar
[INFO] Preparing cobertura:check
[WARNING] Removing: check from forked lifecycle, to prevent recursive
invocation.
[INFO] [resources:resources {execution: default-resources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] Copying 1 resource
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [cobertura:instrument {execution: default}]
[WARNING] No files to instrument.
[INFO] NOT adding cobertura ser file to attached artifacts list.
[INFO] [resources:testResources {execution: default-testResources}]
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] Copying 18 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [surefire:test {execution: default-test}]
D:\Perforce\depot\se\development\services\customer-readonly-service\HEAD\Test\target\surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running TestSuite
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
WARN init, CRITICAL SECURITY CHECKS ARE DISABLED: server SSL certificate
hostname verification is turned off.
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.011 sec
Tests run: 76, Failures: 0, Errors: 0, Skipped: 0
------------------------------
Many thanks for the help
Raluca
--
You received this message because you are subscribed to the Google Groups
"testng-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
Visit this group at http://groups.google.com/group/testng-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.