Prevent leak of URWLOCK_READ_WAITERS flag for urwlocks.

If there was some error, e.g. the sleep was interrupted, as in the
referenced PR, do_rw_rdlock() did not cleared URWLOCK_READ_WAITERS.
Since unlock only wakes up write waiters when there is no read
waiters, for URWLOCK_PREFER_READER kind of locks, the result was
missed wakeups for writers.

In particular, the most visible victims are ld-elf.so locks in
processes which loaded libthr, because rtld locks are urwlocks in
prefer-reader mode.  Normal rwlocks fall into prefer-reader mode only
if thread already owns rw lock in read mode, which is not typical and
correspondingly less visible.  In the PR, unowned rtld bind lock was
waited for in the process where only one thread was left alive.

Note that do_rw_wrlock() correctly clears URWLOCK_WRITE_WAITERS in
case of errors.

Reported and tested by:	longwitz@incore.de
PR:	211947
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week

1 files changed, 7 insertions, 2 deletions

diff --git a/sys/kern/kern_umtx.c b/sys/kern/kern_umtx.c
index 33eb8bca43cc..51495aa23ff5 100644
--- a/sys/kern/kern_umtx.c
+++ b/sys/kern/kern_umtx.c
@@ -2743,9 +2743,12 @@ sleep:
 		suword32(&rwlock->rw_blocked_readers, blocked_readers-1);
 		if (blocked_readers == 1) {
 			rv = fueword32(&rwlock->rw_state, &state);
-			if (rv == -1)
+			if (rv == -1) {
+				umtxq_unbusy_unlocked(&uq->uq_key);
 				error = EFAULT;
-			while (error == 0) {
+				break;
+			}
+			for (;;) {
 				rv = casueword32(&rwlock->rw_state, state,
 				    &oldstate, state & ~URWLOCK_READ_WAITERS);
 				if (rv == -1) {
@@ -2756,6 +2759,8 @@ sleep:
 					break;
 				state = oldstate;
 				error = umtxq_check_susp(td);
+				if (error != 0)
+					break;
 			}
 		}