aboutsummaryrefslogtreecommitdiff
path: root/sys/net
Commit message (Collapse)AuthorAgeFilesLines
* Explicitly change the link state when we assingn an address.Andrey V. Elsukov2018-06-091-0/+1
| | | | | | | | | | | | | | | Since we are setting IFF_UP flag on SIOCSIFADDR, it is possible, that after this link state information still not initialized properly. This leads to problems with routing, since now interface has IFCAP_LINKSTATE capability and a route is considered as working only when interface's link state is in LINK_STATE_UP (see RT_LINK_IS_UP() macro). Reported by: Marek Zarychta MFC after: 3 days Notes: svn path=/head/; revision=334875
* Remove tx task spinning added in r333686Stephen Hurd2018-06-081-10/+0
| | | | | | | | | | | | This caused issues with PASTE. Just remove the reschedule since the DELAY() should be enough for use cases such as pkt-gen which were failing before the change. Reported by: Michio Honda Sponsored by: Limelight Networks Notes: svn path=/head/; revision=334862
* uma: fix up r334824Mateusz Guzik2018-06-081-1/+1
| | | | | | | | | Turns out there is code which ends up passing M_ZERO to counters. Since counters zero unconditionally on their own, just ignore drop the flag in that place. Notes: svn path=/head/; revision=334830
* rtentry_zinit: don't blindly pass through M_ZERO to counter allocMatt Macy2018-06-081-1/+1
| | | | Notes: svn path=/head/; revision=334829
* iflib: Record TCP checksum info in iflib when TCP checksum is requestedEric Joyner2018-06-071-22/+31
| | | | | | | | | | | | | ixl(4) (when it switches over to using iflib) devices need the TCP header length in order to do TCP checksum offload. Reviewed by: gallatin@, shurd@ MFC after: 1 week Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D15558 Notes: svn path=/head/; revision=334779
* Rework if_gif(4) to use new encap_lookup_t method to speedup lookupAndrey V. Elsukov2018-06-052-457/+111
| | | | | | | | | | | | | | | | | | | | | | of needed interface when many gif interfaces are present. Remove rmlock from gif_softc, use epoch(9) and CK_LIST instead. Move more AF-related code into AF-related locations. Use hash table to speedup lookup of needed softc. Interfaces with GIF_IGNORE_SOURCE flag are stored in plain CK_LIST. Sysctl net.link.gif.parallel_tunnels is removed. The removal was planed 16 years ago, and actually it could work only for outbound direction. Each protocol, that can be handled by if_gif(4) interface is registered by separate encap handler, this helps avoid invoking the handler for unrelated protocols (GRE, PIM, etc.). This change allows dramatically improve performance when many gif(4) interfaces are used. Sponsored by: Yandex LLC Notes: svn path=/head/; revision=334673
* Rework IP encapsulation handling code.Andrey V. Elsukov2018-06-055-81/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently it has several disadvantages: - it uses single mutex to protect internal structures. It is used by data- and control- path, thus there are no parallelism at all. - it uses single list to keep encap handlers for both INET and INET6 families. - struct encaptab keeps unneeded information (src, dst, masks, protosw), that isn't used by code in the source tree. - matches are prioritized and when many tunneling interfaces are registered, encapcheck handler of each interface is invoked for each packet. The search takes O(n) for n interfaces. All this work is done with exclusive lock held. What this patch includes: - the datapath is converted to be lockless using epoch(9) KPI. - struct encaptab now linked using CK_LIST. - all unused fields removed from struct encaptab. Several new fields addedr: min_length is the minimum packet length, that encapsulation handler expects to see; exact_match is maximum number of bits, that can return an encapsulation handler, when it wants to consume a packet. - IPv6 and IPv4 handlers are stored in separate lists; - added new "encap_lookup_t" method, that will be used later. It is targeted to speedup lookup of needed interface, when gif(4)/gre(4) have many interfaces. - the need to use protosw structure is eliminated. The only pr_input method was used from this structure, so I don't see the need to keep using it. - encap_input_t method changed to avoid using mbuf tags to store softc pointer. Now it is passed directly trough encap_input_t method. encap_getarg() funtions is removed. - all sockaddr structures and code that uses them removed. We don't have any code in the tree that uses them. All consumers use encap_attach_func() method, that relies on invoking of encapcheck() to determine the needed handler. - introduced struct encap_config, it contains parameters of encap handler that is going to be registered by encap_attach() function. - encap handlers are stored in lists ordered by exact_match value, thus handlers that need more bits to match will be checked first, and if encapcheck method returns exact_match value, the search will be stopped. - all current consumers changed to use new KPI. Reviewed by: mmacy Sponsored by: Yandex LLC Differential Revision: https://reviews.freebsd.org/D15617 Notes: svn path=/head/; revision=334671
* Reduce overhead of entropy collectionMatt Macy2018-05-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | - move harvest mask check inline - move harvest mask to frequently_read out of actively modified cache line - disable ether_input collection and describe its limitations in NOTES Typically entropy collection in ether_input was stirring zero in to the entropy pool while at the same time greatly reducing max pps. This indicates that perhaps we should more closely scrutinize how much entropy we're getting from a given source as well as what our actual entropy collection needs are for seeding Yarrow. Reviewed by: cem, gallatin, delphij Approved by: secteam Differential Revision: https://reviews.freebsd.org/D15526 Notes: svn path=/head/; revision=334450
* Re-apply r190640.Hans Petter Selasky2018-05-311-16/+76
| | | | | | | | | | | | | | | - Restore local change to include <net/bpf.h> inside pcap.h. This fixes ports build problems. - Update local copy of dlt.h with new DLT types. - Revert no longer needed <net/bpf.h> includes which were added as part of r334277. Suggested by: antoine@, delphij@, np@ MFC after: 3 weeks Sponsored by: Mellanox Technologies Notes: svn path=/head/; revision=334418
* if_setlladdr: don't call ioctl in epoch contextMatt Macy2018-05-301-0/+2
| | | | | | | | PR: 228612 Reported by: markj Notes: svn path=/head/; revision=334402
* pf: Add missing include statementKristof Provost2018-05-301-0/+1
| | | | | | | | rmlocks require <sys/lock.h> as well as <sys/rmlock.h>. Unbreak mips build. Notes: svn path=/head/; revision=334379
* pf: Replace rwlock on PF_RULES_LOCK with rmlockKristof Provost2018-05-301-8/+11
| | | | | | | | | | | | | | | Given that PF_RULES_LOCK is a mostly read lock, replace the rwlock with rmlock. This change improves packet processing rate in high pps environments. Benchmarking by olivier@ shows a 65% improvement in pps. While here, also eliminate all appearances of "sys/rwlock.h" includes since it is not used anymore. Submitted by: farrokhi@ Differential Revision: https://reviews.freebsd.org/D15502 Notes: svn path=/head/; revision=334375
* iflib: mark irq allocation name parameter as constantStephen Hurd2018-05-292-13/+13
| | | | | | | | | | | | | | | | | The *name parameter passed to iflib_irq_alloc_generic and iflib_softirq_alloc_generic is never modified. Many places in code pass string literals and thus should not be modified. Mark the *name parameter as a const char * instead, so that we enforce that the name is not modified before passing to bus_describe_intr() Submitted by: Jacob Keller <jacob.e.keller@intel.com> Reviewed by: kmacy Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D15343 Notes: svn path=/head/; revision=334358
* iflib: hold context lock across detach for drivers that need itMatt Macy2018-05-291-3/+5
| | | | Notes: svn path=/head/; revision=334343
* rt_getifa_fib: don't use ifa but info->rti_ifaMatt Macy2018-05-291-1/+1
| | | | | | | Reported by: kp Notes: svn path=/head/; revision=334314
* route: fix missed ref addsMatt Macy2018-05-292-8/+18
| | | | | | | | | - ensure that we bump the ifa ref whenever we add a reference - defer freeing epoch protected references until after the if_purgaddrs loop Notes: svn path=/head/; revision=334305
* iflib: Add new shared flag: IFLIB_ADMIN_ALWAYS_RUNEric Joyner2018-05-262-2/+6
| | | | | | | | | | | | | | | | | | | | ixl(4)'s nvmupdate utility expects the nvmupdate process to run while the interface is down; these nvm update commands use the admin queue, so the admin queue needs to be able to generate interrupts and be processed while the interface is down. So add a flag that ixl(4) sets that lets the entire admin task run even when the interface is marked down/IFF_DRV_RUNNING isn't set. With this change, nvmupdate should function like it did pre-iflib. Reviewed by: gallatin@, sbruno@ MFC after: 1 week Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D15575 Notes: svn path=/head/; revision=334231
* rtrequest1_fib: we need to always bump the ifaddr refcount when we take a ↵Matt Macy2018-05-251-5/+3
| | | | | | | | | | | | | | reference from an rtentry. r334118 introduced a case when this was not done. While we're here make the intent more obvious by moving the refcount bump down to when we know we'll actually need it. Reported by: markj Notes: svn path=/head/; revision=334222
* CK: update consumers to use CK macros across the boardMatt Macy2018-05-245-30/+31
| | | | | | | | r334189 changed the fields to have names distinct from those in queue.h in order to expose the oversights as compile time errors Notes: svn path=/head/; revision=334193
* if_delgroups: add missed unlock introduced by r334118Matt Macy2018-05-241-0/+1
| | | | Notes: svn path=/head/; revision=334172
* UDP: further performance improvements on txMatt Macy2018-05-238-195/+193
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cumulative throughput while running 64 netperf -H $DUT -t UDP_STREAM -- -m 1 on a 2x8x2 SKL went from 1.1Mpps to 2.5Mpps Single stream throughput increases from 910kpps to 1.18Mpps Baseline: https://people.freebsd.org/~mmacy/2018.05.11/udpsender2.svg - Protect read access to global ifnet list with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender3.svg - Protect short lived ifaddr references with epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender4.svg - Convert if_afdata read lock path to epoch https://people.freebsd.org/~mmacy/2018.05.11/udpsender5.svg A fix for the inpcbhash contention is pending sufficient time on a canary at LLNW. Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15409 Notes: svn path=/head/; revision=334118
* Improve MAC address uniqueness on if_epair(4).Luca Pizzamiglio2018-05-231-8/+24
| | | | | | | | | | | | | | | | As reported in PR184149, it can happen that epair devices can have the same MAC address. This solution is based on a 32-bit hash, obtained combining the if_index of the a interface and the hostid. If the hostid is zero, a random number is used. PR: 184149 Reviewed by: wollman, eugen Approved by: cognet Differential Revision: https://reviews.freebsd.org/D15329 Notes: svn path=/head/; revision=334094
* Simplify lagg_input().Mark Johnston2018-05-222-12/+4
| | | | | | | | | No functional change intended. MFC after: 2 weeks Notes: svn path=/head/; revision=334049
* ck: simplify interface with libkvm consumers by defining ck_queue typesMatt Macy2018-05-211-9/+4
| | | | | | | as their queue.h equivalents if !_KERNEL Notes: svn path=/head/; revision=333957
* net: fix uninitialized variable warningMatt Macy2018-05-192-3/+3
| | | | Notes: svn path=/head/; revision=333886
* mp_ring: fix i386Matt Macy2018-05-191-1/+2
| | | | | | | | | | | Even though 64-bit atomics are supported on i386 there are panics indicating that the code does not work correctly there. Switch to mutex based variant (and fix that while we're here). Reported by: pho, kib Notes: svn path=/head/; revision=333879
* net: fix set but not usedMatt Macy2018-05-196-19/+12
| | | | Notes: svn path=/head/; revision=333864
* ifnet: Replace if_addr_lock rwlock with epoch + mutexMatt Macy2018-05-189-69/+104
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Run on LLNW canaries and tested by pho@ gallatin: Using a 14-core, 28-HTT single socket E5-2697 v3 with a 40GbE MLX5 based ConnectX 4-LX NIC, I see an almost 12% improvement in received packet rate, and a larger improvement in bytes delivered all the way to userspace. When the host receiving 64 streams of netperf -H $DUT -t UDP_STREAM -- -m 1, I see, using nstat -I mce0 1 before the patch: InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 4.98 0.00 4.42 0.00 4235592 33 83.80 4720653 2149771 1235 247.32 4.73 0.00 4.20 0.00 4025260 33 82.99 4724900 2139833 1204 247.32 4.72 0.00 4.20 0.00 4035252 33 82.14 4719162 2132023 1264 247.32 4.71 0.00 4.21 0.00 4073206 33 83.68 4744973 2123317 1347 247.32 4.72 0.00 4.21 0.00 4061118 33 80.82 4713615 2188091 1490 247.32 4.72 0.00 4.21 0.00 4051675 33 85.29 4727399 2109011 1205 247.32 4.73 0.00 4.21 0.00 4039056 33 84.65 4724735 2102603 1053 247.32 After the patch InMpps OMpps InGbs OGbs err TCP Est %CPU syscalls csw irq GBfree 5.43 0.00 4.20 0.00 3313143 33 84.96 5434214 1900162 2656 245.51 5.43 0.00 4.20 0.00 3308527 33 85.24 5439695 1809382 2521 245.51 5.42 0.00 4.19 0.00 3316778 33 87.54 5416028 1805835 2256 245.51 5.42 0.00 4.19 0.00 3317673 33 90.44 5426044 1763056 2332 245.51 5.42 0.00 4.19 0.00 3314839 33 88.11 5435732 1792218 2499 245.52 5.44 0.00 4.19 0.00 3293228 33 91.84 5426301 1668597 2121 245.52 Similarly, netperf reports 230Mb/s before the patch, and 270Mb/s after the patch Reviewed by: gallatin Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15366 Notes: svn path=/head/; revision=333813
* epoch(9): allocate net epochs earlier in bootMatt Macy2018-05-181-2/+10
| | | | Notes: svn path=/head/; revision=333809
* epoch: move epoch variables to read mostly sectionMatt Macy2018-05-181-2/+2
| | | | Notes: svn path=/head/; revision=333807
* Use NULL for SYSINIT's last arg, which is a pointer typeEd Maste2018-05-182-2/+2
| | | | | | | Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=333806
* epoch(9): Make epochs non-preemptible by defaultMatt Macy2018-05-183-3/+6
| | | | | | | | | | | There are risks associated with waiting on a preemptible epoch section. Change the name to make them not be the default and document the issue under CAVEATS. Reported by: markj Notes: svn path=/head/; revision=333802
* epoch: add non-preemptible "critical" variantMatt Macy2018-05-181-1/+1
| | | | | | | | | | | | | | | | | adds: - epoch_enter_critical() - can be called inside a different epoch, starts a section that will acquire any MTX_DEF mutexes or do anything that might sleep. - epoch_exit_critical() - corresponding exit call - epoch_wait_critical() - wait variant that is guaranteed that any threads in a section are running. - epoch_global_critical - an epoch_wait_critical safe epoch instance Requested by: markj Approved by: sbruno Notes: svn path=/head/; revision=333775
* Fix !netmap build post r333686Matt Macy2018-05-161-0/+2
| | | | | | | Approved by: sbruno Notes: svn path=/head/; revision=333689
* Work around lack of TX IRQs in iflib for netmapStephen Hurd2018-05-162-7/+35
| | | | | | | | | | | | | | | | | | | | | When poll() is called via netmap, txsync is initially called, and if there are no available buffers to reclaim, it waits for the driver to notify of new buffers. Since the TX IRQ is generally not used in iflib drivers, this ends up causing a timeout. Work around this by having the reclaim DELAY(1) if it's initially unable to reclaim anything, then schedule the tx task, which will spin by continuously rescheduling itself until some buffers are reclaimed. In general, the delay is enough to allow some buffers to be reclaimed, so spinning is minimized. Reported by: Johannes Lundberg <johalun0@gmail.com> Reviewed by: sbruno Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15455 Notes: svn path=/head/; revision=333686
* Replace rmlock with epoch in laggStephen Hurd2018-05-142-112/+109
| | | | | | | | | | | | | | | Use the new epoch based reclamation API. Now the hot paths will not block at all, and the sx lock is used for the softc data. This fixes LORs reported where the rwlock was obtained when the sxlock was held. Submitted by: mmacy Reported by: Harry Schmalzbauer <freebsd@omnilan.de> Reviewed by: sbruno Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D15355 Notes: svn path=/head/; revision=333612
* iflib(9): Add support for cloning pseudo interfacesMatt Macy2018-05-118-62/+915
| | | | | | | | | | | | | | | | | Part 3 of many ... The VPC framework relies heavily on cloning pseudo interfaces (vmnics, vpc switch, vcpswitch port, hostif, vxlan if, etc). This pulls in that piece. Some ancillary changes get pulled in as a side effect. Reviewed by: shurd@ Approved by: sbruno@ Sponsored by: Joyent, Inc. Differential Revision: https://reviews.freebsd.org/D15347 Notes: svn path=/head/; revision=333502
* Apply the change from r272770 to if_ipsec(4) interface.Andrey V. Elsukov2018-05-111-1/+1
| | | | | | | | | | | | | It is guaranteed that if_ipsec(4) interface is used only for tunnel mode IPsec, i.e. decrypted and decapsultaed packet has its own IP header. Thus we can consider it as new packet and clear the protocols flags. This allows ICMP/ICMPv6 properly handle errors that may cause this packet. PR: 228108 MFC after: 1 week Notes: svn path=/head/; revision=333497
* Allow different bridge types to coexistMatt Macy2018-05-115-20/+16
| | | | | | | | | | | | | | if_bridge has a lot of limitations that make it scale poorly to higher data rates. In my projects/VPC branch I leverage the bridge interface between layers for my high speed soft switch as well as for purposes of stacking in general. Reviewed by: sbruno@ Approved by: sbruno@ Differential Revision: https://reviews.freebsd.org/D15344 Notes: svn path=/head/; revision=333481
* Slight cleanup of interface event logging.Dag-Erling Smørgrav2018-05-111-15/+13
| | | | | | | | | | | | | | | | | | | Make if_printf() use vlog() instead of vprintf(). This means it can no longer return the number of characters printed, as it used to, but every single call to if_printf() in the entire kernel ignores the return value anyway; just return 0 so we don't have to change the prototype. Consistently use if_printf() throughout sys/net/if.c, instead of a mixture of if_printf() and log(). In ifa_maintain_loopback_route(), don't needlessly log an error if we either failed to add a route because it already existed or failed to remove one because it did not. We still return an error code, though. MFC after: 1 week Notes: svn path=/head/; revision=333476
* Allocate epoch for networking at startupMatt Macy2018-05-102-0/+5
| | | | | | | | | Additionally add CK to include paths for modules Approved by: sbruno@ Notes: svn path=/head/; revision=333469
* Add IFCAP_LINKSTATE support to if_loop(4).Andrey V. Elsukov2018-05-091-1/+3
| | | | | | | | | | Reviewed by: wollman Obtained from: Yandex LLC MFC after: 3 weeks Differential Revision: https://reviews.freebsd.org/D15278 Notes: svn path=/head/; revision=333400
* iflib: print message when iflib_tx_structures_setup failsStephen Hurd2018-05-081-4/+4
| | | | | | | | | | | | | | | | | | Print a message when iflib_tx_structures_setup fails, like we do for iflib_rx_structures_setup. Now that we always print a message from within iflib_qset_structures_setup when it fails, stop printing one in iflib_device_register() at the call site. Submitted by: Jacob Keller <jacob.e.keller@intel.com> Reviewed by: gallatin MFC after: 3 days Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D15300 Notes: svn path=/head/; revision=333373
* iflib: cleanup queues when iflib_device_register failStephen Hurd2018-05-081-5/+8
| | | | | | | | | | | Submitted by: Jacob Keller <jacob.e.keller@intel.com> Reviewed by: gallatin MFC after: 3 days Sponsored by: Intel Corporation Differential Revision: https://reviews.freebsd.org/D15299 Notes: svn path=/head/; revision=333366
* Fix an off-by-one error when deciding to request a tx interruptAndrew Gallatin2018-05-071-1/+1
| | | | | | | | | | | | | The canonical check for whether or not a ring is drainable is TXQ_AVAIL() > MAX_TX_DESC() + 2. Use this same construct here, in order to avoid a potential off-by-one error where we might otherwise fail to request an interrupt. Reviewed by: mmacy Sponsored by: Netflix Notes: svn path=/head/; revision=333329
* r333175 introduced deferred deletion of multicast addresses in order to ↵Matt Macy2018-05-062-8/+36
| | | | | | | | | | | | | | | | | | permit the driver ioctl to sleep on commands to the NIC when updating multicast filters. More generally this permitted driver's to use an sx as a softc lock. Unfortunately this change introduced a race whereby a a multicast update would still be queued for deletion when ifconfig deleted the interface thus calling down in to _purgemaddrs and synchronously deleting _all_ of the multicast addresses on the interface. Synchronously remove all external references to a multicast address before enqueueing for delete. Reported by: lwhsu Approved by: sbruno Notes: svn path=/head/; revision=333309
* The ifnet pointer (ifp) in rt_newaddrmsg can be valid without ifp->if_addr ↵Matt Macy2018-05-061-1/+4
| | | | | | | | | | | | being set if if the ifnet is still live by way of a reference but in line for deletion. Check ifp->if_addr before dereferencing. Approved by: sbruno Notes: svn path=/head/; revision=333308
* Add netdump support to iflib.Mark Johnston2018-05-061-2/+91
| | | | | | | | | | | | | em(4) and igb(4) were tested by me, and ixgbe(4) and bnxt(4) were tested by sbruno. Reviewed by: mmacy, shurd MFC after: 1 month Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15262 Notes: svn path=/head/; revision=333291
* Import the netdump client code.Mark Johnston2018-05-062-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | This is a component of a system which lets the kernel dump core to a remote host after a panic, rather than to a local storage device. The server component is available in the ports tree. netdump is particularly useful on diskless systems. The netdump(4) man page contains some details describing the protocol. Support for configuring netdump will be added to dumpon(8) in a future commit. To use netdump, the kernel must have been compiled with the NETDUMP option. The initial revision of netdump was written by Darrell Anderson and was integrated into Sandvine's OS, from which this version was derived. Reviewed by: bdrewery, cem (earlier versions), julian, sbruno MFC after: 1 month X-MFC note: use a spare field in struct ifnet Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D15253 Notes: svn path=/head/; revision=333283
* fix gcc8 warningsMatt Macy2018-05-041-14/+20
| | | | | | | Approved by: sbruno Notes: svn path=/head/; revision=333258
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-01 16:39:30 +0000