Draft for splitting embedded tests into several binaries (efr32) #33616
+113
−40
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a proposal for how to split up the tests into different binaries (as suggested by Issue #33360), using efr32 as an example.
It consists of creating more than one
silabs_executable
in thesrc/test_driver/efr32/BUILD.gn
. Each executable has a subset of the tests in it. In this example I make just 2 binaries,efr32_device_tests_bin1
which contains all the tests that are currently enabled for efr32, andefr32_device_tests_bin2
which contains all the tests that are currently disabled by if-statements in the various BUILD.gn files. I haven't yet confirmed that all the currently-disabled tests would fit in bin2, so it might need a 3rd or 4th part.Currently for efr32 there are various if-statements in
src/BUILD.gn
(disabling entire directories of test suites), and insrc/something/tests/BUILD.gn
(disabling specific test suites in that directory). My goal was to start removing the per-platform logic from these BUILD.gn files and instead separate the tests into groups which can be selectively included in the BUILD.gn file for each platform. In this draft i've done it for "efr32", but this process would be repeated for the other platforms.In
src/BUILD.gn
I split the tests up into separate test groups. I chose to split maximally, 1 directory per group. Note that I left the existingchip_test_group("tests")
structure there since all the other platforms are still using it. However I removed the logic relating to efr32. Once the logic for the other embeds is removed, the "tests" group will be used only by the big systems (or the systems have a testrunner that loads the tests one at a time).Then in
src/test_driver/efr32/BUILD.gn
I have asilabs_executable
structure for each binary (in this case 2 binaries) where I add tests on a group-by-group basis to each of the binaries. Some testing will be needed to make sure these balance.TODO: It's possible there are other reasons besides flash size that certain tests were disabled for efr32. If that's the case we'd need to add some conditions back into the
src/something/tests/BUILD.gn
files to disable specific tests that just don't work on efr32.TODO: I use a source_set in order to cut down on repetition in
src/test_driver/efr32/BUILD.gn
, however I'm not sure if i'm using source_set correctly. I need to understand which options should go in the source_set versus going in the silabs_executable for each test binary.TODO: I am not sure yet how to handle
chip_monolithic_tests
option in the embedded tests. efr32 sets this to true.