1) POSIX requires rand(3) return values to be in the [0, RAND_MAX] range,

but ACM formula we use have internal state (and return value) in the
[1, 0x7ffffffe] range, so our RAND_MAX (0x7fffffff) is never reached
because it is off by one, zero is not reached too.

Correct both RAND_MAX and rand(3) return value, shifting last one
to the 0 by 1 subtracted, resulting POSIXed [0, 0x7ffffffd(=new RAND_MAX)]
range.

2) Add a checks for not overflowing on too big seeds. It may happens on
the machines, where sizeof(unsigned int) > 32 bits.

Reviewed by:    bde [1]
MFC after:      2 weeks

1 files changed, 1 insertions, 1 deletions

diff --git a/include/stdlib.h b/include/stdlib.h
index b73a54fa891c..cc922734a829 100644
--- a/include/stdlib.h
+++ b/include/stdlib.h
@@ -69,7 +69,7 @@ typedef struct {
 #define	EXIT_FAILURE	1
 #define	EXIT_SUCCESS	0
 
-#define	RAND_MAX	0x7fffffff
+#define	RAND_MAX	0x7ffffffd
 
 __BEGIN_DECLS
 #ifdef _XLOCALE_H_