| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
MFC after: 1 week
|
|
|
|
|
|
|
| |
Now we can handle EFI runtime service faults on arm64 do so.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46818
|
|
|
|
|
|
|
|
|
|
| |
This is a function that calls into the EFI Runtime Services, but can
handle a fault. To support this add a handler in assembly that can
restore the kernel state on an exception and return a failure to the
caller.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46817
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When we receive an exception while in a spinlock we shouldn't enable
interrupts. When entering a spinlock we disable interrupts so enabling
them here could cause surprising results.
The three cases that could cause this are:
1. A break-before-make sequence
2. Accessing possibly unmapped code with a fault handler
3. Buggy code
1 and 2 are supported later in the data abort handler, and 3 should be
fixed when found.
Reviewed by: mmel, kib, markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46816
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As with other architectures when the TDP_NOFAULTING flag is set we
shouldn't panic when td_critnest is non-zero or if the witness check
fails.
The EFI runtime service functions will soon set this flag to handle
exceptions in the services.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46815
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sys/intr.h originally started life as an extract of arm's intr.h, and
this include was dropped in its place. Changes in flight want to add
some MD definitions that we'll use in the more MI parts of INTRNG.
Let's formally reverse the dependency now since this is way more
common in general. All of the includes switched in this change that I
spot-checked were in-fact wanting declarations historically included in
sys/intr.h anyways.
Reviewed by: andrew, imp, jrtc27, mhorne, mmel, olce
Differential Revision: https://reviews.freebsd.org/D47002
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
uint32_t is handy for directly interfacing with assembly-language. For
the C portion, enum is much handier. In particular there is no need to
count the number of roots by hand. This also works better for being
able to build kernels with varying numbers of roots.
Switch to INTR_ROOT_COUNT as this better matches the purpose of the
value. Switch to root_type, rather than rootnum for similar reasons.
Remove the default from the core. Better to require the architectures
to declare the type since they will routinely deviate and a default
chosen now will likely be suboptimal.
Leave intr_irq_handler() taking a register type as that better matches
for interfacing with assembly-language.
|
|
|
|
|
|
|
| |
Rather than keeping the entire machine interrupt header assembly-safe,
switch to use of the existing genassym.c program to generate them. This
will be handier in the long-term, most of the header never needs to be
exposed to assembly-language.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Debug and trace features such as Statistical Profiling Extension (SPE)
use the CONTEXTIDR_EL1 register to get the PID of the current process.
Add a sysctl switch to toggle writing the current PID into this register
in the thread switcher.
To make use of the feature, the following sysctl switch must be set:
sysctl machdep.pid_in_contextidr=1
Kernel code can also toggle the sysctl by writing directly to the global
var behind the sysctl arm64_pid_in_contextidr:
extern bool arm64_pid_in_contextidr;
Sponsored by: Arm Ltd
|
|
|
|
| |
Sponsored by: Arm Ltd
|
|
|
|
|
|
|
|
|
|
|
|
| |
The ID_AA64MMFR1_EL0.SpecSEI field needs to use MRS_HIGHER to get the
largest value from all CPUs. This is because it indicates when an
exception might happen when it's non-zero and can't happen when zero.
As indicating something that might happen even when it can't is safer
use MRS_HIGHER to handle this field.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47126
|
|
|
|
|
|
|
|
|
|
| |
Use the ID register tables to find how to adjust the ID register fields
in the kernel and vmm views. This allows us to use the same method to
get a common view of CTR_EL0.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47125
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the same method to decode CTR_EL0 as for the ID registers. This
will allow us to create a common view over all CPUs.
This will also allow us to create a common view for userspace and the
kernel if we detect a difference on some CPUs, or to handle errata.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47124
|
|
|
|
|
|
|
|
|
|
|
| |
Some special registers we may want to emulate for userspace have fields
that are reserved to be 1. Add support for these fields.
As there is no value to print jump over printing them in print_id_fields.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47123
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add MRS_EXACT_IF_DIFFERENT, MRS_HIGHER_OR_ZERO, and MRS_HIGHER.
These will be used to handle the cache CTR_EL0 register, and make some
ID register values safer.
They are:
- MRS_EXACT_IF_DIFFERENT: If the field is the same on all CPUs then
use this value, otherwise use the safe value.
- MRS_HIGHER_OR_ZERO: Use the highest value, or zero if seen,
- MRS_HIGHER: Use the highest value.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47122
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When exporting ID registers to userspace we should either adjust them
the same way for FreeBSD and Linux binaries, or export the Linux
field values to a safe value, e.g. when a needed syscall is missing.
To allow for this, and to clean up ID register handling in the kernel
move to using a bit per userspace ABI.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47121
|
|
|
|
|
|
|
|
|
|
| |
To support reworking the arm64 CPU ID code to add CTR_EL0, the cache
type register, start to move the safe value to be encoded as a named
field rather than part of MRS_EXACT.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47120
|
|
|
|
|
|
|
|
|
| |
In preparation for using the ID register decode with the cache type
register support a non-4 bit field width.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47119
|
|
|
|
|
|
|
|
|
| |
In preperation for supporting non-4 bit field width to identcpu.c,
e.g. for ctr_el0, add the width of the existing ID register fields.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47118
|
|
|
|
|
|
|
|
|
| |
The VPIPT and AIVIVT cache types are reserved from Armv8.0. Remove
them as nothing will report these values.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47117
|
|
|
|
|
|
|
|
|
|
|
| |
The VPIPT I-cache policy has been removed from the architecture.
Remove support for it from pmap. The vmm side was not imported so
calls to pmap_invalidate_vpipt_icache would cause the kernel to branch
to a NULL pointer.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D47116
|
|
|
|
|
|
|
| |
While here make it return void, we don't set any useful return value
and nothing checks for it.
Sponsored by: Arm Ltd
|
|
|
|
|
|
|
|
|
| |
All handlers set x0 before returning, there is no need to also do so
in the data abort exception handler.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46814
|
|
|
|
|
|
|
|
|
|
| |
When we get a data abort in an EFI runtime service the userspace pmap
will be locked by the current thread. Skip trying to lock it again as
it will be in a critical section and the lock may sleep.
Reviewed by: markj
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46813
|
|
|
|
|
|
|
|
| |
Simplify the calls into the SMCCC firmware with the new
arm_smccc_invoke* macros.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46987
|
|
|
|
|
|
|
|
|
|
| |
The break-before-make update invalidates PTEs, including the PTE
pointing to curthread, causing a fault in `trace_pc`. This
addresses a similar issue in
01bb9a2a3557bc9389f628d301cd691e08266f1d.
Reviewed by: markj
MFC after: 1 week
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On AmpereOne, the current programming of GITS_BASER2 to individual
allocations per ITS causes a:
panic: APEI Fatal Hardware Error!
This is due to the ITS reporting a BASER2_DATA_ERR error. The GIC-700
expects all GITS_BASER2.Physical_Address fields to match system-wide.
Skip programming any vPE GITS_BASER registers to avoid this crash.
Sponsored by: Ampere Computing LLC
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D46094
|
|
|
|
|
|
|
| |
These add alternative behaviour to some floating-point instructions
so don't need any kernel support and can just be exposed to userspace.
Sponsored by: Arm Ltd
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the FreeBSD/arm64 port was created we only supported FDT. We now
also support ACPI, and have for many years. When this support was
added we kept FDT as the default.
There are some setups where both ACPI tables and a FDT DTB are passed
into the kernel. In most of these cases the DTB is only used to pass
in minimal information.
To handle the cases where both are passed in prefer ACPI over FDT.
Reviewed by: bz, imp, emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46750
|
|
|
|
|
|
|
|
|
| |
Report when SVE is present and allow it to be used by calling
sve_restore_state on an SVE exception from userspace.
Reviewed by: kib
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43310
|
|
|
|
|
|
|
|
|
| |
As with floating point instructions don't trap SVE instructions to the
hypervisor. This lets us handle then in the kernel.
Reviewed by: imp (earlier version)
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43303
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the NT_ARM_SVE note type and use it to access the SVE registers
from ptrace. This allows userspace to modify the full SVE register
values.
Try to follow the Linux semantics to allow debuggers to use this with
minimal changes.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43309
|
|
|
|
|
|
|
|
|
| |
To allow for user space to read the SVE vector length add a sysarch
handler to return the value to userspace.
Reviewed by: imp
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43308
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the new extended register support in the arm64 signal frame to
handle the SVE registers.
As these registers alias the VFP registers we use the floating point
register values to restore the lower bits of the SVE registers. This
is to support software that doesn't understand SVE to continue working.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43307
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add initial kernel support for SVE. This detects if SVE is present on
all CPUs, and if so allows for the use of SVE in the future.
As the SVE registers are a superset of the VFP registers we don't need
to restore the VFP registers when SVE is enabled.
Ths interface to enable SVE is provided, but not used until SVE is
supported in signals and with ptrace.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D43306
|
|
|
|
|
|
|
|
| |
instead of base/size.
Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
|
|
|
|
|
| |
Sponsored by: Advanced Micro Devices (AMD)
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
arm64 supports FIQs, fast interrupt requests, which are required by
Apple silicon which hardwires the ARM timers to FIQs. This is needed by
the upcoming Apple Interrupt Controller. Based on work by andrew@ and
kevans@ in https://reviews.freebsd.org/D40161.
Signed-off-by: Ayrton Munoz <a.munoz3327@gmail.com>
Co-authored-by: Kyle Evans <kevans@FreeBSD.org>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Reviewed-by: imp,mmel,mhorne
Pull-Request: https://github.com/freebsd/freebsd-src/pull/1363
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Different types of interrupts may require using different exception
vectors so this commit adds support multiple interrupt roots to handle
these cases. Archs may opt-in to multiple interrupt roots by defining
INTR_ROOT_NUM as the number of roots in their intr.h. Based off
https://reviews.freebsd.org/D40161.
Signed-off-by: Ayrton Munoz <a.munoz3327@gmail.com>
Co-authored-by: Kyle Evans <kevans@FreeBSD.org>
Co-authored-by: Andrew Turner <andrew@FreeBSD.org>
Reviewed-by: imp,mmel,mhorne
Pull-Request: https://github.com/freebsd/freebsd-src/pull/1363
|
|
|
|
|
|
| |
- s/parametes/parameters/
MFC after: 3 days
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
imx_clk_composite_find_best_parent can fail, when it does for all the
clocks checked we attempt to configure an uninitialized best_parent
leading to a panic.
Initialize best_parent and skip reconfiguring the clock if we don't find
a new best_parent to use.
Reviewed By: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46662
|
|
|
|
|
|
|
|
|
| |
Add clock tree for imx8mp SOC. This provides clocks sufficient for
several sub systems to work including USB and SD/MMC.
Reviewed by: manu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46642
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rearrange the IMX clock control module driver so it is more straight
forward to support clock trees from other SOCs in the family.
Move the existing imx8mq_ccm driver to a more generic imx_ccm (based on
rk_cru) and update the previous driver to sub class imx_ccm.
Reviewed by: manu
Sponsored by: The FreeBSD Foundations
Differential Revision: https://reviews.freebsd.org/D46641
|
|
|
|
|
|
|
|
| |
Add a new macro that enables all CPTR_EL2 traps. This helps ensure we
trap all extensions we don't support.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46516
|
|
|
|
|
|
|
|
| |
The TTA field moves depending on the HCR_EL2.E2H field. Add a macro to
hold the E2H == 1 case.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46515
|
|
|
|
|
|
|
|
| |
Rename register fields that are only valid when HCR_EL2.E2H == 1. Some
fields move around depending on the value of the E2H field.
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46514
|
|
|
|
|
|
| |
Reviewed by: emaste
Sponsored by: Arm Ltd
Differential Revision: https://reviews.freebsd.org/D46513
|