Yesterday, I wanted to gather bandwith usage statistics on my FreeBSD pf(4) based firewall in order to graph incoming and outgoing bandwidth utilization.
pfctl(8) provides the '-s info' flag, which can provide statistics on a single interface at a time. The interface can be chosen with the either the "loginterface" directive in pf.conf, or by using the DIOCSETSTATUSIF ioctl. However, I needed statics for all the network interfaces, not just one.
Fortunately, pf(4) also provides the DIOCIGETIFACES ioctl, which allows me to gather packet and byte statistics on all interfaces at once. This article will provide an introduction to using the pf(4) ioctl interface to gather network interface traffic statistics. Full example source code can be downloaded here. Note that PF does not maintain statistics on traffic that does not pass through PF. If you don't use PF, all the counters will be zero.
pf(4) is controlled via an ioctl interface, using the pseudo-device /dev/pf. A full discussion of the available ioctl commands can be found in the pf(4) man page. This article will only cover gathering of per-interface statistics, but there are many other possibilities, including adding and removing filter rules, anchors, and modifying tables.
In order to operate on /dev/pf with ioctl, we'll need to open the device and acquire a file descriptor; For our purposes, we can open the PF device read-only:
int dev;
dev = open("/dev/pf", OD_RDONLY);
Once we have a handle for /dev/pf, we can issue our ioctl request. The DIOCIGETIFACES ioctl command accepts a pfioc_iface struct as an argument:
struct pfioc_iface {
char pfiio_name[IFNAMSIZ];
void *pfiio_buffer;
int pfiio_esize;
int pfiio_size;
int pfiio_nzero;
int pfiio_flags;
};
According to the pf(4) man page:
If not empty, pfiio_name can be used to restrict the search to a specific interface or driver.
This means that the pfiio_name field can be used specify the interface group (ie, 'dc'), or a specific interface that you are interested in (ie, 'dc3'). If this field is left empty, all interfaces will be returned.
pfiio_buffer[pfiio_size] is the user-supplied buffer for returning the data. On entry, pfiio_size represents the number of pfi_if entries that can fit into the buffer. The kernel will replace this value by the real number of entries it wants to return.
This allows you to allocate a correctly sized buffer, but be aware - if an interface is added between your two ioctl() calls, your buffer will still be incorrectly sized. You must verify that pfiio_size is not larger than the number you originally passed to your final ioctl(2) call.
pfiio_esize should be set to sizeof(struct pfi_if).
This is to ensure that changing the size of the pfi_if struct in future releases will not cause existing code to crash by writing past the length of the allocated buffer. In the kernel, the following check is done:
if (io->pfiio_esize != sizeof(struct pfi_if)) {
error = ENODEV;
break;
}
If the check fails, then ioctl(2) will return -1 and errno will be set to ENODEV. Your application should check the ioctl return value and handle the failure gracefully.
pfiio_flags should be set to PFI_FLAG_GROUP, PFI_FLAG_INSTANCE, or both to tell the kernel to return a group of interfaces (drivers, like "fxp"), real interface instances (like "fxp1") or both.
If pfiio_flags is set to PFI_FLAG_GROUP then only interface driver entries such as fxp or dc will be returned. These entries do not provide any byte or packet statistics.
If pfiio_flags is set to PFI_FLAG_INSTANCE, individual interface statistics will be provided. If used in conjunction with the pfiio_name field, then all interfaces in a specified family can be returned - specifying fxp will return data for fxp0, fxp1, and so on.
Now we can use the ioctl to find the total number of interfaces available:
struct pfioc_iface io; struct pfi_if *ifaces; int numentries = 0;
bzero(&io, sizeof(io));
io.pfiio_buffer = &ifaces; io.pfiio_esize = sizeof(buffer); io.pfiio_size = numentries; io.pfiio_flags = PFI_FLAG_INSTANCE;
if (ioctl(dev, DIOCIGETIFACES, &io) == -1) {
warn("DIOCIGETIFACES failed:");
return (-1);
}
printf("Number of interfaces: %d\n", io.pfiio_size);
The DIOCIGETIFACES ioctl command copies an array of pfi_if structures into the supplied pfioc_iface.pfiio_buffer, and returns the total number of array entries in the pfioc_iface.pfiio_size field.
The pfi_if structure is defined as follows:
struct pfi_if {
char pfif_name[IFNAMSIZ];
u_int64_t pfif_packets[2][2][2];
u_int64_t pfif_bytes[2][2][2];
u_int64_t pfif_addcnt;
u_int64_t pfif_delcnt;
long pfif_tzero;
int pfif_states;
int pfif_rules;
int pfif_flags;
};
The pfif_name field specifies the name of the interface, or the interface group.
The pfif_packets and pfif_bytes fields provide packet and byte counters for both incoming and outgoing traffic. The first array in each field specifies IPv4 or IPv6, the second specifies incoming or outgoing, and the third specifies passed or blocked.
It helps to break it down as follows:
enum { IPV4 = 0, IPV6 = 1};
enum { IN = 0, OUT = 1};
enum { PASS = 0, BLOCK = 1};
u_int64_t ipv4_in_pass = pf_if.pfif_bytes[IPV4][IN][PASS];
The pfif_addcnt and pfif_delcnt fields are used for internal PF reference counting.
The pfif_tzero field specifies the time, in seconds since the UNIX epoch, that the interface was initialized by pf(4). This field will be reset by the DIOCICLRISTATS ioctl.
The pfif_states and pfif_rules fields specify the number of PF states and rules associated with the interface, respectively, and are used for PF interface reference counting.
The pfif_flags field is used to specify a legal combination of the following:
#define PFI_IFLAG_GROUP 0x0001 /* group of interfaces */ #define PFI_IFLAG_INSTANCE 0x0002 /* single instance */ #define PFI_IFLAG_CLONABLE 0x0010 /* clonable group */ #define PFI_IFLAG_DYNAMIC 0x0020 /* dynamic group */ #define PFI_IFLAG_ATTACHED 0x0040 /* interface attached */ #define PFI_IFLAG_REFERENCED 0x0080 /* referenced by rules */
Only the PFI_IFLAG_GROUP and PFI_IFLAG_INSTANCE flags are of particular interest to applications; the remaining flags are used internally by PF. The PFI_IFLAG_GROUP and PFI_IFLAG_INSTANCE flags can be used to discern whether a returned interface is either an interface group or an actual interface.
First, let's write a function that configures the pfioc_iface request structure and provides the returned pfiio_size field to the caller.
int get_pf_ifaces(int dev, const char *filter, struct pfi_if *buffer,
int *numentries, int flags) {
struct pfioc_iface io;
bzero(&io, sizeof(io));
/* If a filter was specified, use it */
if (filter != NULL &&
(strlcpy(io.pfiio_name, filter, sizeof(io.pfiio_name)) >=
sizeof(io.pfiio_name))) {
fprintf(stderr, "Interface string %s too long\n", filter);
return (-1);
}
/* Set up our request structure */ io.pfiio_buffer = buffer; io.pfiio_esize = sizeof(*buffer); io.pfiio_size = *numentries; io.pfiio_flags = flags;
if (ioctl(dev, DIOCIGETIFACES, &io) == -1) {
warn("DIOCIGETIFACES failed:");
return (-1);
}
/* Provide the number of entries to the caller */ *numentries = io.pfiio_size;
return (0); }
Next, let's write a simple printing function that prints a few IPv4 statistics given a pfi_if structure.
void simple_print(struct pfi_if *iface) {
printf("%s IPv4 In: Pass %llu Block %llu\n", iface->pfif_name,
iface->pfif_bytes[IPV4][IN][PASS], iface->pfif_bytes[IPV4][IN][BLOCK]);
printf("%s IPv4 Out: Pass %llu Block %llu\n", iface->pfif_name,
iface->pfif_bytes[IPV4][OUT][PASS], iface->pfif_bytes[IPV4][OUT][BLOCK]);
}
Finally, let's tie it all together with a calling function. I'm going to make this the main() function:
int main(int argc, char *argv[]) {
struct pfi_if *ifaces = NULL;
char *filter= NULL;
int numentries = 0;
int dev, i;
if (argc == 2) {
/* Use an iface filter, if provided */
filter = argv[1];
} else if (argc != 1) {
printf("Usage: %s [interface]\n", argv[0]);
exit (1);
}
dev = open(pf_device, O_RDONLY);
if (dev < 0) {
err(1, "Opening %s failed", pf_device);
}
/* * Get the number of pfi_if structs to be returned and * malloc the required memory */ if(get_pf_ifaces(dev, filter, ifaces, &numentries, PFI_FLAG_INSTANCE) < 0) errx(1, "get_pf_ifaces() failed");
ifaces = xmalloc(sizeof(struct pfi_if) * numentries);
/*
* Record the previous number of entries specified.
* If more are required after the second call, we'll have to realloc
* our ifaces buffer.
*/
i = numentries;
while (1) {
if(get_pf_ifaces(dev, filter, ifaces,
&numentries, PFI_FLAG_INSTANCE) < 0)
errx(1, "get_pf_ifaces() failed");
if (i < numentries) {
/*
* An interface was added prior to this call, and
* after the last get_pf_ifaces call.
* Allocate more space and loop through again.
*/
i = numentries;
xrealloc(ifaces, sizeof(struct pfi_if) * numentries);
} else {
/* The same number or fewer entries were returned */
break;
}
}
for(i = 0; i < numentries; i++) print_iface_stats(&ifaces[i]);
close(dev); exit (0); }
That's it - almost. If you cut and paste this code, it won't compile without the requisite #includes. You can download the code in its functional entirety here.