Page MenuHome

Buildbot: Rerun failed tests with verbose output
AbandonedPublic

Authored by Ankit Meel (ankitm) on Sep 25 2020, 2:11 PM.

Details

Summary

This can help in cases like T81077 where a developer relies
on other platform users to even get a failure log.

Diff Detail

Event Timeline

Ankit Meel (ankitm) requested review of this revision.Sep 25 2020, 2:11 PM
Ankit Meel (ankitm) created this revision.
Sergey Sharybin (sergey) requested changes to this revision.Sep 25 2020, 2:26 PM

I don't think this will work: the buildbot_utils.call() will do sys.exit() if subprocess returns non-zero code.

I'm also not sure whether this is something what is really to be done on the builder, based on:

  • The specific test is easy to reproduce locally.
  • The specific test is intermittent, meaning, running it second time might happen without issues.
  • If some testis only happening in the buildbot, this is up to the buildbot team to look into, who can pass needed information to a developer if the issue is really in the code.
This revision now requires changes to proceed.Sep 25 2020, 2:26 PM
Ankit Meel (ankitm) added a comment.EditedSep 25 2020, 2:57 PM

The specific test is easy to reproduce locally.

The logs in this case would've removed the need to do so. It wouldn't have left Bastien clueless for so long either. 29 July was the first when a failure was spotted.

The specific test is intermittent, meaning, running it second time might happen without issues.

How about uploading the log file Testing/Temporary/LastTest.log contents as another step (by changing master.cfg) ? This will not make the test step's output overly long either.

If some testis only happening in the buildbot, this is up to the buildbot team to look into, who can pass needed information to a developer if the issue is really in the code.

Is more manual work preferred when a machine can do it, and anyone can have a look at the problem? If so, blender.crash.txt (or equivalent for other platforms) and/or Testing/Temporary/LastTest.log should also be passed to the developer by the buildbot team.

In any case, buildbot does not compile with ASAN? so silencing this false positive may 'fix' the issue when running ctest locally on a Mac with an ASAN-enabled build, but I doubt it would help buildbot case at all?

Ankit Meel (ankitm) planned changes to this revision.Sep 25 2020, 3:12 PM

The specific test is easy to reproduce locally.

The logs in this case would've removed the need to do so. It wouldn't have left Bastien clueless for so long either. 29 July was the first when a failure was spotted.

This part of my comment may not be correct if ASan is disabled (which seems so: include("${CMAKE_CURRENT_LIST_DIR}/../../cmake/config/blender_release.cmake").

In this specific case, the log (even the ASAN one) would not be helpful -- they'd just say that there is something wrong happened with memory access somewhere else during the test.
The proper solution to the problem you're trying to solve is to make people to work together. In this case, Bastien should work together with Sebastian B.