MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
/*-
|
|
|
|
|
* Copyright (c) 2015-2016
|
|
|
|
|
* Alexander V. Chernikov <melifaro@FreeBSD.org>
|
|
|
|
|
*
|
|
|
|
|
* Redistribution and use in source and binary forms, with or without
|
|
|
|
|
* modification, are permitted provided that the following conditions
|
|
|
|
|
* are met:
|
|
|
|
|
* 1. Redistributions of source code must retain the above copyright
|
|
|
|
|
* notice, this list of conditions and the following disclaimer.
|
|
|
|
|
* 2. Redistributions in binary form must reproduce the above copyright
|
|
|
|
|
* notice, this list of conditions and the following disclaimer in the
|
|
|
|
|
* documentation and/or other materials provided with the distribution.
|
2017-02-28 23:42:47 +00:00
|
|
|
* 3. Neither the name of the University nor the names of its contributors
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
* may be used to endorse or promote products derived from this software
|
|
|
|
|
* without specific prior written permission.
|
|
|
|
|
*
|
|
|
|
|
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
|
|
|
|
|
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
|
|
|
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
|
|
|
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
|
|
|
|
|
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
|
|
|
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
|
|
|
|
|
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
|
|
|
|
|
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
|
|
|
|
|
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
|
|
|
|
|
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
|
|
|
|
* SUCH DAMAGE.
|
|
|
|
|
*
|
|
|
|
|
* $FreeBSD$
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
#ifndef _NET_ROUTE_VAR_H_
|
|
|
|
|
#define _NET_ROUTE_VAR_H_
|
|
|
|
|
|
2020-04-28 18:42:30 +00:00
|
|
|
#ifndef RNF_NORMAL
|
|
|
|
|
#include <net/radix.h>
|
|
|
|
|
#endif
|
2020-05-23 10:21:02 +00:00
|
|
|
#include <sys/epoch.h>
|
Embed dst sockaddr into rtentry and remove rte packet counter
Currently each rtentry has dst&gateway allocated separately from another zone,
bloating cache accesses.
Current 'struct rtentry' has 12 "mandatory" radix pointers in the beginning,
leaving 4 usable pointers/32 bytes in the first 2 cache lines (amd64).
Fields needed for the datapath are destination sockaddr and rt_nhop.
So far it doesn't look like there is other routable addressing protocol other
than IPv4/IPv6/MPLS, which uses keys longer than 20 bytes.
With that in mind, embed dst into struct rtentry, making the first 24 bytes
of rtentry within 128 bytes. That is enough to make IPv6 address within first
128 bytes.
It is still pretty easy to add code for supporting separately-allocated dst,
however it doesn't make a lot of sense in having such code without a use case.
As rS359823 moved the gateway to the nexthop structure, the dst embedding change
removes the need for any additional allocations done by rt_setgate().
Lastly, as a part of cleanup, remove counter(9) allocation code, as this field
is not used in packet processing anymore.
Reviewed by: ae
Differential Revision: https://reviews.freebsd.org/D24669
2020-05-08 21:06:10 +00:00
|
|
|
#include <netinet/in.h> /* struct sockaddr_in */
|
2020-04-28 18:42:30 +00:00
|
|
|
#include <sys/counter.h>
|
|
|
|
|
|
Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
|
|
|
struct nh_control;
|
|
|
|
|
typedef int rnh_preadd_entry_f_t(u_int fibnum, const struct sockaddr *addr,
|
|
|
|
|
const struct sockaddr *mask, struct nhop_object *nh);
|
|
|
|
|
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
struct rib_head {
|
|
|
|
|
struct radix_head head;
|
|
|
|
|
rn_matchaddr_f_t *rnh_matchaddr; /* longest match for sockaddr */
|
|
|
|
|
rn_addaddr_f_t *rnh_addaddr; /* add based on sockaddr*/
|
|
|
|
|
rn_deladdr_f_t *rnh_deladdr; /* remove based on sockaddr */
|
|
|
|
|
rn_lookup_f_t *rnh_lookup; /* exact match for sockaddr */
|
|
|
|
|
rn_walktree_t *rnh_walktree; /* traverse tree */
|
|
|
|
|
rn_walktree_from_t *rnh_walktree_from; /* traverse tree below a */
|
Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
|
|
|
rnh_preadd_entry_f_t *rnh_preadd; /* hook to alter record prior to insertion */
|
2016-03-24 07:54:56 +00:00
|
|
|
rt_gen_t rnh_gen; /* generation counter */
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
int rnh_multipath; /* multipath capable ? */
|
|
|
|
|
struct radix_node rnh_nodes[3]; /* empty tree for common case */
|
2018-06-16 08:26:23 +00:00
|
|
|
struct rmlock rib_lock; /* config/data path lock */
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
struct radix_mask_head rmhead; /* masks radix head */
|
2020-01-09 17:21:00 +00:00
|
|
|
struct vnet *rib_vnet; /* vnet pointer */
|
|
|
|
|
int rib_family; /* AF of the rtable */
|
|
|
|
|
u_int rib_fibnum; /* fib number */
|
2020-01-22 13:53:18 +00:00
|
|
|
struct callout expire_callout; /* Callout for expiring dynamic routes */
|
|
|
|
|
time_t next_expire; /* Next expire run ts */
|
Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
|
|
|
struct nh_control *nh_control; /* nexthop subsystem data */
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
};
|
|
|
|
|
|
2018-06-16 08:26:23 +00:00
|
|
|
#define RIB_RLOCK_TRACKER struct rm_priotracker _rib_tracker
|
|
|
|
|
#define RIB_LOCK_INIT(rh) rm_init(&(rh)->rib_lock, "rib head lock")
|
|
|
|
|
#define RIB_LOCK_DESTROY(rh) rm_destroy(&(rh)->rib_lock)
|
|
|
|
|
#define RIB_RLOCK(rh) rm_rlock(&(rh)->rib_lock, &_rib_tracker)
|
|
|
|
|
#define RIB_RUNLOCK(rh) rm_runlock(&(rh)->rib_lock, &_rib_tracker)
|
|
|
|
|
#define RIB_WLOCK(rh) rm_wlock(&(rh)->rib_lock)
|
|
|
|
|
#define RIB_WUNLOCK(rh) rm_wunlock(&(rh)->rib_lock)
|
|
|
|
|
#define RIB_LOCK_ASSERT(rh) rm_assert(&(rh)->rib_lock, RA_LOCKED)
|
|
|
|
|
#define RIB_WLOCK_ASSERT(rh) rm_assert(&(rh)->rib_lock, RA_WLOCKED)
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
|
2020-05-04 14:31:45 +00:00
|
|
|
/* Constants */
|
|
|
|
|
#define RIB_MAX_RETRIES 3
|
|
|
|
|
|
2020-01-21 12:00:34 +00:00
|
|
|
/* Macro for verifying fields in af-specific 'struct route' structures */
|
|
|
|
|
#define CHK_STRUCT_FIELD_GENERIC(_s1, _f1, _s2, _f2) \
|
|
|
|
|
_Static_assert(sizeof(((_s1 *)0)->_f1) == sizeof(((_s2 *)0)->_f2), \
|
|
|
|
|
"Fields " #_f1 " and " #_f2 " size differs"); \
|
|
|
|
|
_Static_assert(__offsetof(_s1, _f1) == __offsetof(_s2, _f2), \
|
|
|
|
|
"Fields " #_f1 " and " #_f2 " offset differs");
|
|
|
|
|
|
|
|
|
|
#define _CHK_ROUTE_FIELD(_route_new, _field) \
|
|
|
|
|
CHK_STRUCT_FIELD_GENERIC(struct route, _field, _route_new, _field)
|
|
|
|
|
|
|
|
|
|
#define CHK_STRUCT_ROUTE_FIELDS(_route_new) \
|
2020-04-25 09:06:11 +00:00
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_nh) \
|
2020-01-21 12:00:34 +00:00
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_lle) \
|
|
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_prepend)\
|
|
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_plen) \
|
|
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_flags) \
|
|
|
|
|
_CHK_ROUTE_FIELD(_route_new, ro_mtu) \
|
|
|
|
|
_CHK_ROUTE_FIELD(_route_new, spare)
|
|
|
|
|
|
|
|
|
|
#define CHK_STRUCT_ROUTE_COMPAT(_ro_new, _dst_new) \
|
|
|
|
|
CHK_STRUCT_ROUTE_FIELDS(_ro_new); \
|
|
|
|
|
_Static_assert(__offsetof(struct route, ro_dst) == __offsetof(_ro_new, _dst_new),\
|
|
|
|
|
"ro_dst and " #_dst_new " are at different offset")
|
|
|
|
|
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
struct rib_head *rt_tables_get_rnh(int fib, int family);
|
2020-04-11 07:37:08 +00:00
|
|
|
void rt_mpath_init_rnh(struct rib_head *rnh);
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
|
Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
|
|
|
VNET_PCPUSTAT_DECLARE(struct rtstat, rtstat);
|
|
|
|
|
#define RTSTAT_ADD(name, val) \
|
|
|
|
|
VNET_PCPUSTAT_ADD(struct rtstat, rtstat, name, (val))
|
|
|
|
|
#define RTSTAT_INC(name) RTSTAT_ADD(name, 1)
|
|
|
|
|
|
2020-04-28 18:42:30 +00:00
|
|
|
struct rtentry {
|
|
|
|
|
struct radix_node rt_nodes[2]; /* tree glue, and other values */
|
|
|
|
|
/*
|
|
|
|
|
* XXX struct rtentry must begin with a struct radix_node (or two!)
|
|
|
|
|
* because the code does some casts of a 'struct radix_node *'
|
|
|
|
|
* to a 'struct rtentry *'
|
|
|
|
|
*/
|
|
|
|
|
#define rt_key(r) (*((struct sockaddr **)(&(r)->rt_nodes->rn_key)))
|
|
|
|
|
#define rt_mask(r) (*((struct sockaddr **)(&(r)->rt_nodes->rn_mask)))
|
|
|
|
|
#define rt_key_const(r) (*((const struct sockaddr * const *)(&(r)->rt_nodes->rn_key)))
|
|
|
|
|
#define rt_mask_const(r) (*((const struct sockaddr * const *)(&(r)->rt_nodes->rn_mask)))
|
Embed dst sockaddr into rtentry and remove rte packet counter
Currently each rtentry has dst&gateway allocated separately from another zone,
bloating cache accesses.
Current 'struct rtentry' has 12 "mandatory" radix pointers in the beginning,
leaving 4 usable pointers/32 bytes in the first 2 cache lines (amd64).
Fields needed for the datapath are destination sockaddr and rt_nhop.
So far it doesn't look like there is other routable addressing protocol other
than IPv4/IPv6/MPLS, which uses keys longer than 20 bytes.
With that in mind, embed dst into struct rtentry, making the first 24 bytes
of rtentry within 128 bytes. That is enough to make IPv6 address within first
128 bytes.
It is still pretty easy to add code for supporting separately-allocated dst,
however it doesn't make a lot of sense in having such code without a use case.
As rS359823 moved the gateway to the nexthop structure, the dst embedding change
removes the need for any additional allocations done by rt_setgate().
Lastly, as a part of cleanup, remove counter(9) allocation code, as this field
is not used in packet processing anymore.
Reviewed by: ae
Differential Revision: https://reviews.freebsd.org/D24669
2020-05-08 21:06:10 +00:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* 2 radix_node structurs above consists of 2x6 pointers, leaving
|
|
|
|
|
* 4 pointers (32 bytes) of the second cache line on amd64.
|
|
|
|
|
*
|
|
|
|
|
*/
|
2020-04-28 18:42:30 +00:00
|
|
|
struct nhop_object *rt_nhop; /* nexthop data */
|
Embed dst sockaddr into rtentry and remove rte packet counter
Currently each rtentry has dst&gateway allocated separately from another zone,
bloating cache accesses.
Current 'struct rtentry' has 12 "mandatory" radix pointers in the beginning,
leaving 4 usable pointers/32 bytes in the first 2 cache lines (amd64).
Fields needed for the datapath are destination sockaddr and rt_nhop.
So far it doesn't look like there is other routable addressing protocol other
than IPv4/IPv6/MPLS, which uses keys longer than 20 bytes.
With that in mind, embed dst into struct rtentry, making the first 24 bytes
of rtentry within 128 bytes. That is enough to make IPv6 address within first
128 bytes.
It is still pretty easy to add code for supporting separately-allocated dst,
however it doesn't make a lot of sense in having such code without a use case.
As rS359823 moved the gateway to the nexthop structure, the dst embedding change
removes the need for any additional allocations done by rt_setgate().
Lastly, as a part of cleanup, remove counter(9) allocation code, as this field
is not used in packet processing anymore.
Reviewed by: ae
Differential Revision: https://reviews.freebsd.org/D24669
2020-05-08 21:06:10 +00:00
|
|
|
union {
|
|
|
|
|
/*
|
|
|
|
|
* Destination address storage.
|
|
|
|
|
* sizeof(struct sockaddr_in6) == 28, however
|
|
|
|
|
* the dataplane-relevant part (e.g. address) lies
|
|
|
|
|
* at offset 8..24, making the address not crossing
|
|
|
|
|
* cacheline boundary.
|
|
|
|
|
*/
|
|
|
|
|
struct sockaddr_in rt_dst4;
|
|
|
|
|
struct sockaddr_in6 rt_dst6;
|
|
|
|
|
struct sockaddr rt_dst;
|
|
|
|
|
char rt_dstb[28];
|
|
|
|
|
};
|
|
|
|
|
|
2020-04-28 18:42:30 +00:00
|
|
|
int rt_flags; /* up/down?, host/net */
|
|
|
|
|
u_long rt_weight; /* absolute weight */
|
|
|
|
|
u_long rt_expire; /* lifetime for route, e.g. redirect */
|
Embed dst sockaddr into rtentry and remove rte packet counter
Currently each rtentry has dst&gateway allocated separately from another zone,
bloating cache accesses.
Current 'struct rtentry' has 12 "mandatory" radix pointers in the beginning,
leaving 4 usable pointers/32 bytes in the first 2 cache lines (amd64).
Fields needed for the datapath are destination sockaddr and rt_nhop.
So far it doesn't look like there is other routable addressing protocol other
than IPv4/IPv6/MPLS, which uses keys longer than 20 bytes.
With that in mind, embed dst into struct rtentry, making the first 24 bytes
of rtentry within 128 bytes. That is enough to make IPv6 address within first
128 bytes.
It is still pretty easy to add code for supporting separately-allocated dst,
however it doesn't make a lot of sense in having such code without a use case.
As rS359823 moved the gateway to the nexthop structure, the dst embedding change
removes the need for any additional allocations done by rt_setgate().
Lastly, as a part of cleanup, remove counter(9) allocation code, as this field
is not used in packet processing anymore.
Reviewed by: ae
Differential Revision: https://reviews.freebsd.org/D24669
2020-05-08 21:06:10 +00:00
|
|
|
#define rt_endzero rt_mtx
|
2020-04-28 18:42:30 +00:00
|
|
|
struct mtx rt_mtx; /* mutex for routing entry */
|
|
|
|
|
struct rtentry *rt_chain; /* pointer to next rtentry to delete */
|
2020-05-23 10:21:02 +00:00
|
|
|
struct epoch_context rt_epoch_ctx; /* net epoch tracker */
|
2020-04-28 18:42:30 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
#define RT_LOCK_INIT(_rt) \
|
|
|
|
|
mtx_init(&(_rt)->rt_mtx, "rtentry", NULL, MTX_DEF | MTX_DUPOK | MTX_NEW)
|
|
|
|
|
#define RT_LOCK(_rt) mtx_lock(&(_rt)->rt_mtx)
|
|
|
|
|
#define RT_UNLOCK(_rt) mtx_unlock(&(_rt)->rt_mtx)
|
|
|
|
|
#define RT_LOCK_DESTROY(_rt) mtx_destroy(&(_rt)->rt_mtx)
|
|
|
|
|
#define RT_LOCK_ASSERT(_rt) mtx_assert(&(_rt)->rt_mtx, MA_OWNED)
|
|
|
|
|
#define RT_UNLOCK_COND(_rt) do { \
|
|
|
|
|
if (mtx_owned(&(_rt)->rt_mtx)) \
|
|
|
|
|
mtx_unlock(&(_rt)->rt_mtx); \
|
|
|
|
|
} while (0)
|
|
|
|
|
|
Introduce nexthop objects and new routing KPI.
This is the foundational change for the routing subsytem rearchitecture.
More details and goals are available in https://reviews.freebsd.org/D24141 .
This patch introduces concept of nexthop objects and new nexthop-based
routing KPI.
Nexthops are objects, containing all necessary information for performing
the packet output decision. Output interface, mtu, flags, gw address goes
there. For most of the cases, these objects will serve the same role as
the struct rtentry is currently serving.
Typically there will be low tens of such objects for the router even with
multiple BGP full-views, as these objects will be shared between routing
entries. This allows to store more information in the nexthop.
New KPI:
struct nhop_object *fib4_lookup(uint32_t fibnum, struct in_addr dst,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
struct nhop_object *fib6_lookup(uint32_t fibnum, const struct in6_addr *dst6,
uint32_t scopeid, uint32_t flags, uint32_t flowid);
These 2 function are intended to replace all all flavours of
<in_|in6_>rtalloc[1]<_ign><_fib>, mpath functions and the previous
fib[46]-generation functions.
Upon successful lookup, they return nexthop object which is guaranteed to
exist within current NET_EPOCH. If longer lifetime is desired, one can
specify NHR_REF as a flag and get a referenced version of the nexthop.
Reference semantic closely resembles rtentry one, allowing sed-style conversion.
Additionally, another 2 functions are introduced to support uRPF functionality
inside variety of our firewalls. Their primary goal is to hide the multipath
implementation details inside the routing subsystem, greatly simplifying
firewalls implementation:
int fib4_lookup_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
int fib6_lookup_urpf(uint32_t fibnum, const struct in6_addr *dst6, uint32_t scopeid,
uint32_t flags, const struct ifnet *src_if);
All functions have a separate scopeid argument, paving way to eliminating IPv6 scope
embedding and allowing to support IPv4 link-locals in the future.
Structure changes:
* rtentry gets new 'rt_nhop' pointer, slightly growing the overall size.
* rib_head gets new 'rnh_preadd' callback pointer, slightly growing overall sz.
Old KPI:
During the transition state old and new KPI will coexists. As there are another 4-5
decent-sized conversion patches, it will probably take a couple of weeks.
To support both KPIs, fields not required by the new KPI (most of rtentry) has to be
kept, resulting in the temporary size increase.
Once conversion is finished, rtentry will notably shrink.
More details:
* architectural overview: https://reviews.freebsd.org/D24141
* list of the next changes: https://reviews.freebsd.org/D24232
Reviewed by: ae,glebius(initial version)
Differential Revision: https://reviews.freebsd.org/D24232
2020-04-12 14:30:00 +00:00
|
|
|
/*
|
|
|
|
|
* With the split between the routing entry and the nexthop,
|
|
|
|
|
* rt_flags has to be split between these 2 entries. As rtentry
|
|
|
|
|
* mostly contains prefix data and is thought to be generic enough
|
|
|
|
|
* so one can transparently change the nexthop pointer w/o requiring
|
|
|
|
|
* any other rtentry changes, most of rt_flags shifts to the particular nexthop.
|
|
|
|
|
* /
|
|
|
|
|
*
|
|
|
|
|
* RTF_UP: rtentry, as an indication that it is linked.
|
|
|
|
|
* RTF_HOST: rtentry, nhop. The latter indication is needed for the datapath
|
|
|
|
|
* RTF_DYNAMIC: nhop, to make rtentry generic.
|
|
|
|
|
* RTF_MODIFIED: nhop, to make rtentry generic. (legacy)
|
|
|
|
|
* -- "native" path (nhop) properties:
|
|
|
|
|
* RTF_GATEWAY, RTF_STATIC, RTF_PROTO1, RTF_PROTO2, RTF_PROTO3, RTF_FIXEDMTU,
|
|
|
|
|
* RTF_PINNED, RTF_REJECT, RTF_BLACKHOLE, RTF_BROADCAST
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/* Nexthop rt flags mask */
|
|
|
|
|
#define NHOP_RT_FLAG_MASK (RTF_GATEWAY | RTF_HOST | RTF_REJECT | RTF_DYNAMIC | \
|
|
|
|
|
RTF_MODIFIED | RTF_STATIC | RTF_BLACKHOLE | RTF_PROTO1 | RTF_PROTO2 | \
|
|
|
|
|
RTF_PROTO3 | RTF_FIXEDMTU | RTF_PINNED | RTF_BROADCAST)
|
|
|
|
|
|
|
|
|
|
/* rtentry rt flag mask */
|
|
|
|
|
#define RTE_RT_FLAG_MASK (RTF_UP | RTF_HOST)
|
|
|
|
|
|
|
|
|
|
/* Nexthop selection */
|
|
|
|
|
#define _NH2MP(_nh) ((struct nhgrp_object *)(_nh))
|
|
|
|
|
#define _SELECT_NHOP(_nh, _flowid) \
|
|
|
|
|
(_NH2MP(_nh))->nhops[(_flowid) % (_NH2MP(_nh))->mp_size]
|
|
|
|
|
#define _RT_SELECT_NHOP(_nh, _flowid) \
|
|
|
|
|
((!NH_IS_MULTIPATH(_nh)) ? (_nh) : _SELECT_NHOP(_nh, _flowid))
|
|
|
|
|
#define RT_SELECT_NHOP(_rt, _flowid) _RT_SELECT_NHOP((_rt)->rt_nhop, _flowid)
|
|
|
|
|
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
/* rte<>nhop translation */
|
|
|
|
|
static inline uint16_t
|
|
|
|
|
fib_rte_to_nh_flags(int rt_flags)
|
|
|
|
|
{
|
|
|
|
|
uint16_t res;
|
|
|
|
|
|
|
|
|
|
res = (rt_flags & RTF_REJECT) ? NHF_REJECT : 0;
|
2019-05-08 23:39:24 +00:00
|
|
|
res |= (rt_flags & RTF_HOST) ? NHF_HOST : 0;
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
res |= (rt_flags & RTF_BLACKHOLE) ? NHF_BLACKHOLE : 0;
|
|
|
|
|
res |= (rt_flags & (RTF_DYNAMIC|RTF_MODIFIED)) ? NHF_REDIRECT : 0;
|
|
|
|
|
res |= (rt_flags & RTF_BROADCAST) ? NHF_BROADCAST : 0;
|
|
|
|
|
res |= (rt_flags & RTF_GATEWAY) ? NHF_GATEWAY : 0;
|
|
|
|
|
|
|
|
|
|
return (res);
|
|
|
|
|
}
|
|
|
|
|
|
2020-01-22 13:53:18 +00:00
|
|
|
void tmproutes_update(struct rib_head *rnh, struct rtentry *rt);
|
|
|
|
|
void tmproutes_init(struct rib_head *rh);
|
|
|
|
|
void tmproutes_destroy(struct rib_head *rh);
|
MFP r287070,r287073: split radix implementation and route table structure.
There are number of radix consumers in kernel land (pf,ipfw,nfs,route)
with different requirements. In fact, first 3 don't have _any_ requirements
and first 2 does not use radix locking. On the other hand, routing
structure do have these requirements (rnh_gen, multipath, custom
to-be-added control plane functions, different locking).
Additionally, radix should not known anything about its consumers internals.
So, radix code now uses tiny 'struct radix_head' structure along with
internal 'struct radix_mask_head' instead of 'struct radix_node_head'.
Existing consumers still uses the same 'struct radix_node_head' with
slight modifications: they need to pass pointer to (embedded)
'struct radix_head' to all radix callbacks.
Routing code now uses new 'struct rib_head' with different locking macro:
RADIX_NODE_HEAD prefix was renamed to RIB_ (which stands for routing
information base).
New net/route_var.h header was added to hold routing subsystem internal
data. 'struct rib_head' was placed there. 'struct rtentry' will also
be moved there soon.
2016-01-25 06:33:15 +00:00
|
|
|
|
|
|
|
|
#endif
|
