From 89b57fcf01d895a82a24b97548116e924bca5efc Mon Sep 17 00:00:00 2001 From: Konstantin Belousov Date: Mon, 5 Nov 2007 11:36:16 +0000 Subject: Fix for the panic("vm_thread_new: kstack allocation failed") and silent NULL pointer dereference in the i386 and sparc64 pmap_pinit() when the kmem_alloc_nofault() failed to allocate address space. Both functions now return error instead of panicing or dereferencing NULL. As consequence, vmspace_exec() and vmspace_unshare() returns the errno int. struct vmspace arg was added to vm_forkproc() to avoid dealing with failed allocation when most of the fork1() job is already done. The kernel stack for the thread is now set up in the thread_alloc(), that itself may return NULL. Also, allocation of the first process thread is performed in the fork1() to properly deal with stack allocation failure. proc_linkup() is separated into proc_linkup() called from fork1(), and proc_linkup0(), that is used to set up the kernel process (was known as swapper). In collaboration with: Peter Holm Reviewed by: jhb --- sys/i386/ibcs2/imgact_coff.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'sys/i386/ibcs2/imgact_coff.c') diff --git a/sys/i386/ibcs2/imgact_coff.c b/sys/i386/ibcs2/imgact_coff.c index b078e5654972..74a01f4c4a9c 100644 --- a/sys/i386/ibcs2/imgact_coff.c +++ b/sys/i386/ibcs2/imgact_coff.c @@ -337,7 +337,9 @@ exec_coff_imgact(imgp) VOP_UNLOCK(imgp->vp, 0, td); - exec_new_vmspace(imgp, &ibcs2_svr3_sysvec); + error = exec_new_vmspace(imgp, &ibcs2_svr3_sysvec); + if (error) + goto fail; vmspace = imgp->proc->p_vmspace; for (i = 0; i < nscns; i++) { -- cgit v1.2.3