F5 – RST or ICMP Packet Rate

You can follow SOL13151 in order to increase the packets/sec value. However, I would caution against doing it or at least recommend keeping the value smaller. The default setting is in place to prevent the F5 from overwhelming its resources by sending out RST. This could potentially end up being a self-inflicted DoS. So, either don’t change the value or increase the value in minimal steps like +50 to 300 packets/sec.

tmsh
modify sys db tm.maxrejectrate value 300
save sys config

You may have to use SOL13223 in order to identify the reason for RST. SOL9812 provides reasons for which the F5 sends RST.

In order to understand which monitor is marking the pool member down, see this SOL13898. If you are using 11.4 and after code version, the monitor that triggered the failure should be auto-displayed as per K14407

K12531 is a good reference for troubleshooting monitors in F5.

F5 iControl REST

F5 utilizes iControl REST API as part of their automation toolkit. REST API is a powerful way to automate F5 management. iControl REST API was introduced by F5 in 11.5 code version. 11.6 code version is the first major code version with a relatively stable release. However, 11.6 does not support remote authentication like TACACS+. For iControl REST API with remove authentication, it is important to utilize 12.x code version. F5 programmability training documentation and related information are available here.

GTM Code Upgrade

These are a few quick checks as part of the GTM code upgrade maintenance that will be useful.  As part of the maintenance preparatory work, check the license “service check date” as per K7727

Before starting the code upgrade and after the code upgrade, the following can be utilized to check the status of the devices:

From tmsh:

(/Common)(tmos)# show sys software
(/Common)(tmos)# show gtm server | grep -e "Gtm::" -e "Availability" -e "State"

From bash:

/shared/bin/big3d –v

From another client machine:

dig @<GTM1_IP> <WIP_FQDN>
dig @<GTM2_IP> <WIP_FQDN>

Just after the code is upgraded, make sure to run the big3d_install commands as per K13312. This will help to make sure that all the devices run the latest big3d version.

Network Troubleshooting Tools

This is a short list of commonly available network troubleshooting tools that has been of use to me:

  1. Ping.
  2. traceroute.
  3. nslookup/dig
  4. curl.
  5. Error logs on the device.
  6. tcpdump.
  7. Developer tools within a browser like Chrome/FF.
  8. Wireshark.

 

 

F5 GTM – DNS Query Processing Order

When a DNS query arrives at a F5 GTM/DNS, this is the processing order for the DNS query.

1 – DNS Query is processed by the Listener.

2- If Recursion Desired (RD) flag is set in the incoming query and if the DNS Profile associated with the Listener has “Process Recursion Desired” enabled, the following is done:

a. DNS iRule

b. DNSSEC Key Processing

c. DNS Express

d. DNS Profiles

3 – If Recursion Desired (RD) flag is set in the incoming query and if the DNS Profile associated with the Listener has “Process Recursion Desired” disabled, the query is considered “Un-handled” and dispatched according to “Unhanded Query Action” set in DNS Profile.

4 – DNS Cache is used to handle any DNS query that doesn’t match Big-IP GTM/DNS or DNS Express Records.

Reference: K14510

Ansible Playbook Optimizing

$ cat ansible.cfg 
[defaults]
hostfile = ./hosts
host_key_checking = False
timeout = 5 
log_path = ./logfile.txt
forks = 50
gathering = smart

[ssh_connection]
pipelining = True

The above file shows the content of ansible.cfg file. I have added the following to make my playbook run faster:

forks
gathering
pipelining

Forks indicate the number of parallel processes spawned to communicate with remote hosts. Default forks is 5 in Ansible.

Gathering indicates the default policy for fact gathering. When “gather_facts” is True within the playbook, facts are gathered for each host. The facts associated with each host will be discovered only once even when the host is referred in multiple plays when we use “smart” within the ansible.cfg file.

Pipelining enabled will reduce the number of SSH operations required to execute a module on a remote host.

Ansible & Python 3

While using Ansible 2.2 and Python 3.x, I ran into the following error:

ERROR! Unexpected Exception: No module named 'urlparse'

Workaround: 

1. Downgrade Python to 2.x

2. If there is no root level permission, use virtual env:

virtualenv --python=python2.7

iRule HTTP Referer

This is a simple iRule that performs redirects based on HTTP Referer:

when HTTP_REQUEST { 
if { ([HTTP::header exists "Referer"]) and 
    ([URI::host [HTTP::header value Referer]] eq"special.com") } {
HTTP::redirect https://www.site.com/special-client/
} else {
HTTP::redirect https://www.site.com/generic-client
}
}

Ansible Components

What?

Ansible is a simple IT automation tool. Ansible exists as CLI & GUI. GUI is called the Ansible Tower and Ansible, Inc., which is owned by RedHat, officially supports this.

Controlling Nodes:

The Network infrastructure is managed from these Controlling Nodes. In an Enterprise environment, Controlling Nodes are typically Linux bastion servers.

Managed Nodes:

Managed Nodes are the Network Devices that is being managed by the Controlling Nodes. Managed Nodes are typically of Cisco, Juniper, and Arista make and can be classified as Switches, Routers, Firewalls and Load Balancers based on their function from a Network Engineer’s perspective.

Why?

There are many automation tools like Chef, Puppet, and CFEngine but in my opinion, Ansible is suited for Network Automation for the following reasons:

  1. Ansible does not require an agent to be installed in the Managed Node (Network Device).
  2. Ansible requires Python on the Managed Node and most Network Devices support Python.
  3. Ansible relies on YAML as the descriptive language and Jinja2 for templates.

Among the points mentioned above, most Network Vendors do not support the installation of agents and even if they did support the installation, it would be tough to get the relevant permissions within an organization to install the agents in an Enterprise environment that has different Network Teams managing different aspects of the infrastructure.

Fortunately, most network vendors provide native support for Python and Ansible rely on this to execute automation tasks on the “Managed Nodes”.

As a Network Engineer working in an environment with significant scale (1,000s of Network Devices across multiple datacenters), Ansible has been quite useful in obtaining data and deploying configuration. Ansible seems to have widespread support among the Network Engineers seeking automation to manage at scale and there are resources online that can be leveraged to implement Network Automation Solutions.

Ansible Components

Ansible requires the following components in order to automate Network Infrastructure:

  • Controlling Nodes
  • Managed Nodes
  • Ansible Playbook

As noted earlier, Controlling Nodes are usually Linux Bastion Servers that are used to access the switches/routers and other Network Devices. These Network Devices are referred to as the Managed Nodes. Managed Nodes are stored in the hosts file for Ansible automation.

Ansible Playbook:

Ansible Playbooks are expressed in YAML format and serve as the repository for the various tasks that will be executed on the Managed Nodes (hosts). Playbooks are a collection of tasks that will be run on one or more hosts.

Setting up Ansible:

After installing Ansible, I recommend creating a separate directory from which Ansible is executed. For this process, let’s create a directory named “AnsiblePlay”. Within the “AnsiblePlay” directory, I will have the following files:

  • ansible.cfg
  • hosts

and the following directories ./AnsiblePlay/

  • templates
  • hosts_var

Ansible Configuration File:

An Ansible Playbook utilizes the Ansible Configuration File in order to access resources required for the Ansible Playbook. For example, the configuration file stores location information for the hosts file that contains the Managed Nodes (hosts) on which the playbook is executed.

Ansible Configuration File can exist in the following locations and is utilized by the Ansible playbook in the following order.

* ANSIBLE_CONFIG (an environment variable)
* ansible.cfg (in the current directory)
* .ansible.cfg (in the home directory)
* /etc/ansible/ansible.cfg

I would recommend creating your own Ansible configuration file in the Ansible directory. I use the following:

$ cat ansible.cfg

[defaults]
hostfile = ./hosts
host_key_checking=False
timeout = 5

Inventory File or Hosts File:

Inventory File or Hosts File is a text file that contains the Managed Nodes (Hosts) that will be subjected to automation tasks defined in the playbook.

Inventory File can be static or dynamic. For now, the examples use static inventory files.

This is an inventory/host file example:

$ cat hosts
[ios]
Switch-1
Switch-2
Switch-3
Switch-4

[distro]
Distro1
Distro2

[aggr]
Aggr1
Aggr2

Lists: [ ]

Dictionaries: { }. Dictionary has “Key: Value” pair.

YAML – Anything is a string. Quoting strings is optional most of the times.

FACTS:

FACTS are data about the Managed Nodes. Example: Code Version running on Managed Node.

 TASK:

Ansible playbook contains one or more tasks. A task makes sure that the hosts exist in a specific state. When there are multiple tasks and if any task fails, subsequent tasks will not be executed.

 Idempotent:

Running a task once or multiple times is the same in terms of the final output. For example, a task that involves creating a user in the Managed Node will create the user only once no matter how many times the task is executed.