aboutsummaryrefslogtreecommitdiff
path: root/contrib/kyua
Commit message (Collapse)AuthorAgeFilesLines
* kyua: Pass unprivileged user config prop to ATF using all known namesIgor Ostapenko12 days4-3/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Kyua and ATF speak different naming styles. In this case, the unprivileged user property can be named with underscore on the Kyua side, and with a hyphen on the ATF side. Sometimes it is not obvious which style should be used in which situation. For instance, a test case may require this configuration property being set using require.config. Also, a test case may want to read the property using something like atf_tc_get_config_var(). Which names should be used in these cases? From the perspective of the original code, it is expected to be this: require.config unprivileged-user atf_tc_get_config_var(tc, "unprivileged-user") But, as long as Kyua is the main interface, its users expect to work with kyua.conf(5), which says that it must be named as unprivileged_user (with underscore). As a result, test authors tend to do this instead: require.config unprivileged_user atf_tc_get_config_var(tc, "unprivileged_user") Kyua already has hacks to understand both unprivileged_user and unprivileged-user coming from require.config. And this patch covers the missing second part -- make Kyua pass both names back to ATF as two identical configuration properties named different ways. Reviewed by: ngie, asomers MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D49039
* kyua: Support require.diskspace ATF metadataIgor Ostapenko2025-02-023-1/+27
| | | | | | Reviewed by: ngie MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D48333
* kyuafile.5: ATF metadata mapping referenceIgor Ostapenko2025-01-261-1/+44
| | | | | | Reviewed by: ngie MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D48190
* kyua: Support is_exclusive metadata coming from an ATF test caseIgor Ostapenko2024-12-242-0/+35
| | | | | | | | On ATF side it is named "is.exclusive". Reviewed by: ngie, markj MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D47671
* kyua: Do not count skipped as passed in test cmdIgor Ostapenko2024-10-054-48/+107
| | | | | | | | | | | | | | | | | | | | | | | | | | | It changes output of 'kyua test' CLI command only. Hence, other outputs like junit are kept intact for CI and other use cases. It's meant to improve UX of attended use cases. The issue is that the following can be tricky to interpret: 222/222 passed (0 failed) It can be read as all tests are passed, but it might be a summary line of all tests skipped due to some requirement is not met. It's reworked to easily distinguish such cases: 222/222 passed (0 broken, 0 failed, 0 skipped) 0/222 passed (0 broken, 0 failed, 222 skipped) The overall formula is: <actually passed>/<total> (<details about not actually passed ones>) Suggested by: kp Reviewed by: ngie, markj Approved by: markj (mentor) Differential Revision: https://reviews.freebsd.org/D46653
* kyua: fix gcc buildsBrooks Davis2024-07-193-0/+4
| | | | | | | | | | | For some reason execenv::exec() isn't successfully marked noreturn (rlibby reports that virtual functions can't be noreturn), but calling methods are so gcc rightly complains. Work around this by adding explicit __builtin_unreachable() calls. Reviewed by: imp, markj Fixes: 257e70f1d5ee kyua: Add FreeBSD Jail execution environment support Differential Revision: https://reviews.freebsd.org/D46041
* kyua: Add FreeBSD Jail execution environment supportIgor Ostapenko2024-07-1839-27/+1778
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A new Kyua concept is added -- "execution environment". A test can be configured to be run within a specific environment. The test case lifecycle is extended respectively: - execenv init (creates a jail or does nothing for default execenv="host") - test exec - cleanup exec (optional) - execenv cleanup (removes a jail or does nothing for default execenv="host") The following new functionality is provided, from bottom to top: 1 ATF based tests - The new "execenv" metadata property can be set to explicitly ask for an execution environment: "host" or "jail". If it's not defined, as all existing tests do, then it implicitly means "host". - The new "execenv.jail.params" metadata property can be optionally defined to ask Kyua to use specific jail(8) parameters during creation of a temporary jail. An example is "vnet allow.raw_sockets". Kyua implicitly adds "children.max" to "execenv_jail_params" parameters with the maximum possible value. A test case can override it. 2 Kyuafile - The same new metadata properties can be defined on Kyuafile level: "execenv" and "execenv_jail_params". - Note that historically ATF uses dotted style of metadata naming, while Kyua uses underscore style. Hence "execenv.jail.params" vs. "execenv_jail_params". 3 kyua.conf, kyua CLI - The new "execenvs" engine configuration variable can be set to a list of execution environments to run only tests designed for. Tests of not listed environments are skipped. - By default, this variable lists all execution environments supported by a Kyua binary, e.g. execenvs="host jail". - This variable can be changed via "kyua.conf" or via kyua CLI's "-v" parameter. For example, "kyua -v execenvs=host test" will run only host-based tests and skip jail-based ones. - Current value of this variable can be examined with "kyua config". [markj] This feature has not landed upstream yet. See the discussion in https://github.com/freebsd/kyua/pull/224 . Having the ability to automatically jail tests allows many network tests to run in parallel, giving a drastic speedup. So, let's import the feature and start using it in main. Signed-off-by: Igor Ostapenko <pm@igoro.pro> Reviewed by: markj, kp Tested by: markj, kp MFC after: 3 months Differential Revision: https://reviews.freebsd.org/D45865
* contrib/kyua: Merge vendor importMuhammad Moinur Rahman2024-01-1825-31/+304
| | | | | | | | * commit '5f174897f67783925f4ec69122673f9bad6ee6fe': vendor/kyua: Update to snapshot 84c8ec8 Vendor import of freebsd/kyua@a0d44bb356e0c816 Approved by: markj Differential Revision: https://reviews.freebsd.org/D43475
* Revert r359385-r359387Enji Cooper2020-03-2914-3472/+0
| | | | | | | | | | As noted by brooks/emaste, this is the wrong approach to take. Revert the changes so brooks can apply a more proper change. Requested by: brooks, emaste Notes: svn path=/head/; revision=359417
* Follow up to r359385Enji Cooper2020-03-2814-0/+3472
| | | | | | | | | Actually add the generated manpages to unbreak the build. MFC with: r359385 Notes: svn path=/head/; revision=359386
* Import the kyua test framework.Brooks Davis2020-03-23542-0/+96704
Having kyua in the base system will simplify automated testing in CI and eliminates bootstrapping issues on new platforms. The build of kyua is controlled by WITH(OUT)_TESTS_SUPPORT. Reviewed by: emaste Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D24103 Notes: svn path=/head/; revision=359260
generated by cgit v1.2.3 (git 2.25.1) at 2025-03-07 01:18:11 +0000