Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft for splitting embedded tests into several binaries (efr32) #33616

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

feasel0
Copy link
Contributor

@feasel0 feasel0 commented May 27, 2024

This is a proposal for how to split up the tests into different binaries (as suggested by Issue #33360), using efr32 as an example.

It consists of creating more than one silabs_executable in the src/test_driver/efr32/BUILD.gn. Each executable has a subset of the tests in it. In this example I make just 2 binaries, efr32_device_tests_bin1 which contains all the tests that are currently enabled for efr32, and efr32_device_tests_bin2 which contains all the tests that are currently disabled by if-statements in the various BUILD.gn files. I haven't yet confirmed that all the currently-disabled tests would fit in bin2, so it might need a 3rd or 4th part.

Currently for efr32 there are various if-statements in src/BUILD.gn (disabling entire directories of test suites), and in src/something/tests/BUILD.gn (disabling specific test suites in that directory). My goal was to start removing the per-platform logic from these BUILD.gn files and instead separate the tests into groups which can be selectively included in the BUILD.gn file for each platform. In this draft i've done it for "efr32", but this process would be repeated for the other platforms.

In src/BUILD.gn I split the tests up into separate test groups. I chose to split maximally, 1 directory per group. Note that I left the existing chip_test_group("tests") structure there since all the other platforms are still using it. However I removed the logic relating to efr32. Once the logic for the other embeds is removed, the "tests" group will be used only by the big systems (or the systems have a testrunner that loads the tests one at a time).

Then in src/test_driver/efr32/BUILD.gn I have a silabs_executable structure for each binary (in this case 2 binaries) where I add tests on a group-by-group basis to each of the binaries. Some testing will be needed to make sure these balance.

TODO: It's possible there are other reasons besides flash size that certain tests were disabled for efr32. If that's the case we'd need to add some conditions back into the src/something/tests/BUILD.gn files to disable specific tests that just don't work on efr32.

TODO: I use a source_set in order to cut down on repetition in src/test_driver/efr32/BUILD.gn, however I'm not sure if i'm using source_set correctly. I need to understand which options should go in the source_set versus going in the silabs_executable for each test binary.

TODO: I am not sure yet how to handle chip_monolithic_tests option in the embedded tests. efr32 sets this to true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment