2 .\" Copyright (c) 2009, Sun Microsystems, Inc. All Rights Reserved
3 .\" The contents of this file are subject to the terms of the Common Development and Distribution License (the "License"). You may not use this file except in compliance with the License.
4 .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE or http://www.opensolaris.org/os/licensing. See the License for the specific language governing permissions and limitations under the License.
5 .\" When distributing Covered Code, include this CDDL HEADER in each file and include the License file at usr/src/OPENSOLARIS.LICENSE. If applicable, add the following below this CDDL HEADER, with the fields enclosed by brackets "[]" replaced with your own identifying information: Portions Copyright [yyyy] [name of copyright owner]
6 .TH FLOWADM 8 "April 9, 2016"
8 flowadm \- administer bandwidth resource control and priority for protocols,
9 services, containers, and virtual machines
13 \fBflowadm show-flow\fR [\fB-pP\fR] [\fB-S\fR] [\fB-s\fR [\fB-i\fR \fIinterval\fR]] [\fB-l\fR \fIlink\fR]
14 [\fB-o\fR \fIfield\fR[,...]] [\fIflow\fR]
19 \fBflowadm add-flow\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-l\fR \fIlink\fR \fB-a\fR \fIattr\fR=\fIvalue\fR[,...]
20 \fB-p\fR \fIprop\fR=\fIvalue\fR[,...] \fIflow\fR
21 \fBflowadm remove-flow\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] {\fB-l\fR \fIlink\fR | \fIflow\fR}
26 \fBflowadm set-flowprop\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-p\fR \fIprop\fR=\fIvalue\fR[,...] \fIflow\fR
27 \fBflowadm reset-flowprop\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] [\fB-p\fR \fIprop\fR[,...]] \fIflow\fR
28 \fBflowadm show-flowprop\fR [\fB-cP\fR] [\fB-l\fR \fIlink\fR] [\fB-o\fR \fIfield\fR[,...]]
29 [\fB-p\fR \fIprop\fR[,...]] [\fIflow\fR]
34 \fBflowadm show-usage\fR [\fB-a\fR] [\fB-d\fR | {\fB-p\fR \fIplotfile\fR \fB-F\fR \fIformat\fR}] [\fB-s\fR \fItime\fR]
35 [\fB-e\fR \fItime\fR] \fB-f\fR \fIfilename\fR [\fIflow\fR]
40 The \fBflowadm\fR command is used to create, modify, remove, and show
41 networking bandwidth and associated resources for a type of traffic on a
45 The \fBflowadm\fR command allows users to manage networking bandwidth resources
46 for a transport, service, or a subnet. The service is specified as a
47 combination of transport and local port. The subnet is specified by its IP
48 address and subnet mask. The command can be used on any type of data link,
49 including physical links, virtual NICs, and link aggregations.
52 A flow is defined as a set of attributes based on Layer 3 and Layer 4 headers,
53 which can be used to identify a protocol, service, or a virtual machine. When a
54 flow is identified based on flow attributes, separate kernel resources
55 including layer 2, 3, and 4 queues, their processing threads, and other
56 resources are uniquely created for it, such that other traffic has minimal or
60 Inbound and outbound packet are matched to flows in a very fast and scalable
61 way, so that limits can be enforced with minimal performance impact.
64 The \fBflowadm\fR command can be used to identify a flow without imposing any
65 bandwidth resource control. This would result in the traffic type getting its
66 own resources and queues so that it is isolated from rest of the networking
67 traffic for more observable and deterministic behavior.
70 \fBflowadm\fR is implemented as a set of subcommands with corresponding
71 options. Options are described in the context of each subcommand.
74 The following subcommands are supported:
78 \fB\fBflowadm show-flow\fR [\fB-pP\fR] [\fB-s\fR [\fB-i\fR \fIinterval\fR]]
79 [\fB-o\fR \fIfield\fR[,...]] [\fB-l\fR \fIlink\fR] [\fIflow\fR]\fR
83 Show flow configuration information (the default) or statistics, either for all
84 flows, all flows on a link, or for the specified \fIflow\fR.
88 \fB\fB-o\fR \fIfield\fR[,...]\fR
92 A case-insensitive, comma-separated list of output fields to display. The field
93 name must be one of the fields listed below, or a special value \fBall\fR, to
94 display all fields. For each flow found, the following fields can be displayed:
102 The name of the flow.
112 The name of the link the flow is on.
122 IP address of the flow. This can be either local or remote depending on how the
129 \fB\fBtransport\fR\fR
133 The name of the layer for protocol to be used.
143 Local port of service for flow.
153 Differentiated services value for flow and mask used with \fBDSFIELD\fR value
154 to state the bits of interest in the differentiated services field of the IP
163 \fB\fB-p\fR, \fB--parsable\fR\fR
167 Display using a stable machine-parsable format.
173 \fB\fB-P\fR, \fB--persistent\fR\fR
177 Display persistent flow property information.
183 \fB\fB-S\fR, \fB--continuous\fR\fR
187 Continuously display network utilization by flow in a manner similar to the way
188 that \fBprstat\fR(8) displays CPU utilization by process.
194 \fB\fB-s\fR, \fB--statistics\fR\fR
198 Displays flow statistics.
204 \fB\fB-i\fR \fIinterval\fR, \fB--interval\fR=\fIinterval\fR\fR
208 Used with the \fB-s\fR option to specify an interval, in seconds, at which
209 statistics should be displayed. If this option is not specified, statistics are
216 \fB\fB-l\fR \fIlink\fR, \fB--link\fR=\fIlink\fR | \fIflow\fR\fR
220 Display information for all flows on the named link or information for the
229 \fB\fBflowadm add-flow\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-l\fR
230 \fIlink\fR \fB-a\fR \fIattr\fR=\fIvalue\fR[,...] \fB-p\fR
231 \fIprop\fR=\fIvalue\fR[,...] \fIflow\fR\fR
235 Adds a flow to the system. The flow is identified by its flow attributes and
238 As part of identifying a particular flow, its bandwidth resource can be limited
239 and its relative priority to other traffic can be specified. If no bandwidth
240 limit or priority is specified, the traffic still gets its unique layer 2, 3,
241 and 4 queues and processing threads, including NIC hardware resources (when
242 supported), so that the selected traffic can be separated from others and can
243 flow with minimal impact from other traffic.
247 \fB\fB-t\fR, \fB--temporary\fR\fR
251 The changes are temporary and will not persist across reboots. Persistence is
258 \fB\fB-R\fR \fIroot-dir\fR, \fB--root-dir\fR=\fIroot-dir\fR\fR
262 Specifies an alternate root directory where \fBflowadm\fR should apply
269 \fB\fB-l\fR \fIlink\fR, \fB--link\fR=\fIlink\fR\fR
273 Specify the link to which the flow will be added.
279 \fB\fB-a\fR \fIattr\fR=\fIvalue\fR[,...], \fB--attr\fR=\fIvalue\fR\fR
283 A comma-separated list of attributes to be set to the specified values.
289 \fB\fB-p\fR \fIprop\fR=\fIvalue\fR[,...], \fB--prop\fR=\fIvalue\fR[,...]\fR
293 A comma-separated list of properties to be set to the specified values.
301 \fB\fBflowadm remove-flow\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-l\fR
302 {\fIlink\fR | \fIflow\fR}\fR
306 Remove an existing flow identified by its link or name.
310 \fB\fB-t\fR, \fB--temporary\fR\fR
314 The changes are temporary and will not persist across reboots. Persistence is
321 \fB\fB-R\fR \fIroot-dir\fR, \fB--root-dir\fR=\fIroot-dir\fR\fR
325 Specifies an alternate root directory where \fBflowadm\fR should apply
332 \fB\fB-l\fR \fIlink\fR | \fIflow\fR, \fB--link\fR=\fIlink\fR | \fIflow\fR\fR
336 If a link is specified, remove all flows from that link. If a single flow is
337 specified, remove only that flow.
345 \fB\fBflowadm set-flowprop\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-p\fR
346 \fIprop\fR=\fIvalue\fR[,...] \fIflow\fR\fR
350 Set values of one or more properties on the flow specified by name. The
351 complete list of properties can be retrieved using the \fBshow-flow\fR
356 \fB\fB-t\fR, \fB--temporary\fR\fR
360 The changes are temporary and will not persist across reboots. Persistence is
367 \fB\fB-R\fR \fIroot-dir\fR, \fB--root-dir\fR=\fIroot-dir\fR\fR
371 Specifies an alternate root directory where \fBflowadm\fR should apply
372 persistent setting of properties.
378 \fB\fB-p\fR \fIprop\fR=\fIvalue\fR[,...], \fB--prop\fR=\fIvalue\fR[,...]\fR
382 A comma-separated list of properties to be set to the specified values.
390 \fB\fBflowadm reset-flowprop\fR [\fB-t\fR] [\fB-R\fR \fIroot-dir\fR] \fB-p\fR
391 [\fIprop\fR=\fIvalue\fR[,...]] \fIflow\fR\fR
395 Resets one or more properties to their default values on the specified flow. If
396 no properties are specified, all properties are reset. See the
397 \fBshow-flowprop\fR subcommand for a description of properties, which includes
398 their default values.
402 \fB\fB-t\fR, \fB--temporary\fR\fR
406 Specifies that the resets are temporary. Temporary resets last until the next
413 \fB\fB-R\fR \fIroot-dir\fR, \fB--root-dir\fR=\fIroot-dir\fR\fR
417 Specifies an alternate root directory where \fBflowadm\fR should apply
418 persistent setting of properties.
424 \fB\fB-p\fR \fIprop\fR=\fIvalue\fR[,...], \fB--prop\fR=\fIvalue\fR[,...]\fR
428 A comma-separated list of properties to be reset.
436 \fB\fBflowadm show-flowprop\fR [\fB-cP\fR] [\fB-l\fR \fIlink\fR] [\fB-p\fR
437 \fIprop\fR[,...]] [\fIflow\fR]\fR
441 Show the current or persistent values of one or more properties, either for all
442 flows, flows on a specified link, or for the specified flow.
444 By default, current values are shown. If no properties are specified, all
445 available flow properties are displayed. For each property, the following
446 fields are displayed:
454 The name of the flow.
464 The name of the property.
474 The current (or persistent) property value. The value is shown as \fB--\fR
475 (double hyphen), if it is not set, and \fB?\fR (question mark), if the value is
476 unknown. Persistent values that are not set or have been reset will be shown as
477 \fB--\fR and will use the system \fBDEFAULT\fR value (if any).
487 The default value of the property. If the property has no default value,
488 \fB--\fR (double hyphen), is shown.
498 A comma-separated list of the values the property can have. If the values span
499 a numeric range, the minimum and maximum values might be shown as shorthand. If
500 the possible values are unknown or unbounded, \fB--\fR (double hyphen), is
504 Flow properties are documented in the "Flow Properties" section, below.
512 Display using a stable machine-parsable format.
518 \fB\fB-P\fR, \fB--persistent\fR\fR
522 Display persistent flow property information.
528 \fB\fB-p\fR \fIprop\fR[,...], \fB--prop\fR=\fIprop\fR[,...]\fR
532 A comma-separated list of properties to show.
540 \fB\fBflowadm show-usage\fR [\fB-a\fR] [\fB-d\fR | {\fB-p\fR \fIplotfile\fR
541 \fB-F\fR \fIformat\fR}] [\fB-s\fR \fItime\fR] [\fB-e\fR \fItime\fR]
546 Show the historical network flow usage from a stored extended accounting file.
547 Configuration and enabling of network accounting through \fBacctadm\fR(8) is
548 required. The default output will be the summary of flow usage for the entire
549 period of time in which extended accounting was enabled.
557 Display all historical network usage for the specified period of time during
558 which extended accounting is enabled. This includes the usage information for
559 the flows that have already been deleted.
569 Display the dates for which there is logging information. The date is in the
570 format \fIDD\fR/\fIMM\fR/\fIYYYY\fR.
576 \fB\fB-F\fR \fIformat\fR\fR
580 Specifies the format of \fIplotfile\fR that is specified by the \fB-p\fR
581 option. As of this release, \fBgnuplot\fR is the only supported format.
587 \fB\fB-p\fR \fIplotfile\fR\fR
591 When specified with \fB-s\fR or \fB-e\fR (or both), outputs flow usage data to
592 a file of the format specified by the \fB-F\fR option, which is required.
598 \fB\fB-s\fR \fItime\fR, \fB-e\fR \fItime\fR\fR
602 Start and stop times for data display. Time is in the format
603 \fIYYYY\fR.\fIMM\fR.\fIDD\fR,\fIhh\fR:\fImm\fR:\fIss\fR.
609 \fB\fB-f\fR \fIfilename\fR\fR
613 Read extended accounting records of network flow usage from \fIfilename\fR.
623 If specified, display the network flow usage only from the named flow.
624 Otherwise, display network usage from all flows.
629 .SS "Flow Attributes"
631 The flow operand that identify a flow in a \fBflowadm\fR command is a
632 comma-separated list of one or more keyword, value pairs from the list below.
636 \fB\fBlocal_ip\fR[\fB/\fR\fIprefix_len\fR]\fR
640 Identifies a network flow by the local IP address. \fIvalue\fR must be a IPv4
641 address in dotted-decimal notation or an IPv6 address in colon-separated
642 notation. \fIprefix_len\fR is optional.
644 If \fIprefix_len\fR is specified, it describes the netmask for a subnet
645 address, following the same notation convention of \fBifconfig\fR(8) and
646 \fBroute\fR(8) addresses. If unspecified, the given IP address will be
647 considered as a host address for which the default prefix length for a IPv4
648 address is \fB/32\fR and for IPv6 is \fB/128\fR.
654 \fB\fBremote_ip\fR[\fB/\fR\fIprefix_len\fR]\fR
658 Identifies a network flow by the remote IP address. The syntax is the same as
659 \fBlocal_ip\fR attributes
665 \fB\fBtransport\fR={\fBtcp\fR|\fBudp\fR|\fBsctp\fR|\fBicmp\fR|\fBicmpv6\fR}\fR
669 Identifies a layer 4 protocol to be used. It is typically used in combination
670 with local_port to identify the service that needs special attention.
676 \fB\fBlocal_port\fR\fR
680 Identifies a service specified by the local port.
686 \fB\fBdsfield\fR[\fB:\fR\fIdsfield_mask\fR]\fR
690 Identifies the 8-bit differentiated services field (as defined in RFC 2474).
692 The optional \fIdsfield_mask\fR is used to state the bits of interest in the
693 differentiated services field when comparing with the \fBdsfield\fR value. A
694 \fB0\fR in a bit position indicates that the bit value needs to be ignored and
695 a \fB1\fR indicates otherwise. The mask can range from \fB0x01\fR to
696 \fB0xff\fR. If \fIdsfield_mask\fR is not specified, the default mask \fB0xff\fR
697 is used. Both the \fBdsfield\fR value and mask must be in hexadecimal.
702 The following five types of combinations of attributes are supported:
706 local_ip[/\fIprefixlen\fR]=\fIaddress\fR
707 remote_ip[/\fIprefixlen\fR]=\fIaddress\fR
708 transport={tcp|udp|sctp|icmp|icmpv6}
709 transport={tcp|udp|sctp},local_port=\fIport\fR
710 dsfield=\fIval\fR[:\fIdsfield_mask\fR]
717 On a given link, the combinations above are mutually exclusive. An attempt to
718 create flows of different combinations will fail.
721 There are individual flow restrictions and flow restrictions per zone.
722 .SS "Individual Flow Restrictions"
724 Restrictions on individual flows do not require knowledge of other flows that
725 have been added to the link.
728 An attribute can be listed only once for each flow. For example, the following
729 command is not valid:
733 # \fBflowadm add-flow -l vnic1 -a local_port=80,local_port=8080 httpflow\fR
740 \fBtransport\fR and \fBlocal_port\fR:
743 TCP, UDP, or SCTP flows can be specified with a local port. An ICMP or ICMPv6
744 flow that specifies a port is not allowed. The following commands are valid:
748 # \fBflowadm add-flow -l e1000g0 -a transport=udp udpflow\fR
749 # \fBflowadm add-flow -l e1000g0 -a transport=tcp,local_port=80 \e
757 The following commands are not valid:
761 # \fBflowadm add-flow -l e1000g0 -a local_port=25 flow25\fR
762 # \fBflowadm add-flow -l e1000g0 -a transport=icmpv6,local_port=16 \e
768 .SS "Flow Restrictions Per Zone"
770 Within a zone, no two flows can have the same name. After adding a flow with
771 the link specified, the link will not be required for display, modification, or
772 deletion of the flow.
773 .SS "Flow Properties"
775 The following flow properties are supported. Note that the ability to set a
776 given property to a given value depends on the driver and hardware.
784 Sets the full duplex bandwidth for the flow. The bandwidth is specified as an
785 integer with one of the scale suffixes(\fBK\fR, \fBM\fR, or \fBG\fR for Kbps,
786 Mbps, and Gbps). If no units are specified, the input value will be read as
787 Mbps. The default is no bandwidth limit.
797 Sets the relative priority for the flow. The value can be given as one of the
798 tokens \fBhigh\fR, \fBmedium\fR, or \fBlow\fR. The default is \fBmedium\fR.
803 \fBExample 1 \fRCreating a Policy Around a Mission-Critical Port
806 The command below creates a policy around inbound HTTPS traffic on an HTTPS
807 server so that HTTPS obtains dedicated NIC hardware and kernel TCP/IP
808 resources. The name specified, \fBhttps-1\fR, can be used later to modify or
814 # \fBflowadm add-flow -l bge0 -a transport=TCP,local_port=443 https-1\fR
815 # \fBflowadm show-flow -l bge0\fR
816 FLOW LINK IP ADDR PROTO PORT DSFLD
817 https1 bge0 -- tcp 443 --
823 \fBExample 2 \fRModifying an Existing Policy to Add Bandwidth Resource Control
826 The following command modifies the \fBhttps-1\fR policy from the preceding
827 example. The command adds bandwidth control and give the policy a high
833 # \fBflowadm set-flowprop -p maxbw=500M,priority=high https-1\fR
834 # \fBflowadm show-flow https-1\fR
835 FLOW LINK IP ADDR PROTO PORT DSFLD
836 https1 bge0 -- tcp 443 --
838 # \fBflowadm show-flowprop https-1\fR
839 FLOW PROPERTY VALUE DEFAULT POSSIBLE
840 https-1 maxbw 500 -- --
841 https-1 priority HIGH -- LOW,NORMAL,HIGH
847 \fBExample 3 \fRLimiting the UDP Bandwidth Usage
850 The following command creates a policy for UDP protocol so that it cannot
851 consume more than 100Mbps of available bandwidth. The flow is named
857 # \fBflowadm add-flow -l bge0 -a transport=UDP -p maxbw=100M, \e
858 priority=low limit-udp-1\fR
864 \fBExample 4 \fRShowing Flow Usage
867 Flow usage statistics can be stored using the extended accounting facility,
873 # \fBacctadm -e extended -f /var/log/net.log net\fR
876 Network accounting: active
877 Network accounting file: /var/log/net.log
878 Tracked Network resources: extended
879 Untracked Network resources: none
886 The historical data that was saved can be retrieved in summary form using the
887 \fBshow-usage\fR subcommand of \fBflowadm\fR.
890 \fBExample 5 \fRSetting Policy, Making Use of \fBdsfield\fR Attribute
893 The following command sets a policy for EF PHB (DSCP value of 101110 from RFC
894 2598) with a bandwidth of 500 Mbps and a high priority. The \fBdsfield\fR value
895 for this flow will be \fB0x2e\fR (101110) with the \fBdsfield_mask\fR being
896 \fB0xfc\fR (because we want to ignore the 2 least significant bits).
901 # \fBflowadm add-flow -l bge0 -a dsfield=0x2e:0xfc \e
902 -p maxbw=500M,priority=high efphb-flow\fR
909 Display summary information:
914 # \fBflowadm show-usage -f /var/log/net.log\fR
915 FLOW DURATION IPACKETS RBYTES OPACKETS OBYTES BANDWIDTH
916 flowtcp 100 1031 546908 0 0 43.76 Kbps
917 flowudp 0 0 0 0 0 0.00 Mbps
924 Display dates for which logging information is available:
929 # \fBflowadm show-usage -d -f /var/log/net.log\fR
937 Display logging information for \fBflowtcp\fR starting at 02/19/2008, 10:38:46
938 and ending at 02/19/2008, 10:40:06:
943 # \fBflowadm show-usage -s 02/19/2008,10:39:06 -e 02/19/2008,10:40:06 \e
944 -f /var/log/net.log flowtcp\fR
945 FLOW TIME IPACKETS RBYTES OPACKETS OBYTES BANDWIDTH
946 flowtcp 10:39:06 1 1546 4 6539 3.23 Kbps
947 flowtcp 10:39:26 2 3586 5 9922 5.40 Kbps
948 flowtcp 10:39:46 1 240 1 216 182.40 bps
949 flowtcp 10:40:06 0 0 0 0 0.00 bps
956 Output the same information as above as a plotfile:
961 # \fBflowadm show-usage -s 02/19/2008,10:39:06 -e 02/19/2008,10:40:06 \e
962 -p /home/plot/myplot -F gnuplot -f /var/log/net.log flowtcp\fR
963 # \fBTime tcp-flow\fR
979 All actions were performed successfully.
994 See \fBattributes\fR(5) for descriptions of the following attributes:
1002 ATTRIBUTE TYPE ATTRIBUTE VALUE
1004 Interface Stability Committed
1009 \fBacctadm\fR(8), \fBdladm\fR(8), \fBifconfig\fR(8), \fBprstat\fR(8),
1010 \fBroute\fR(8), \fBattributes\fR(5), \fBdlpi\fR(7P)