Tag: Brocade ADX

Brocade ADX Source NAT

Global source NAT

Similar to F5’s Automap

Automap will SNAT any traffic going towards the real-server. The client’s source IP will be replaced with the self-IP configured on the Brocade’s interface closest to the real servers. SNAT is implemented for ALL the real servers.

server source-nat

 

Warning: Do not use automap in environments with any considerable number of clients and/or servers, since there is a high chance to run into port-exhaustion and connections will drop.

SNAT IP

server source-nat 
server source-nat-ip 192.168.100.100 255.255.255.255 0.0.0.0 port-range 2 port-alloc-per-real

The client’s source IP will be replaced with the 192.168.100.100 configured in the second line.

The port-alloc-per-real command indicates that a SNAT IP:port combination can be re-used per real server at any particular instance. The  port-range  parameter specifies which port range this peer uses for source NAT for this source IP address. Specify 1 for the lower port range or 2 for the upper port range.

Per-real-server source NAT

SNAT IP is the IP of the interface closest to the real servers.

SNAT is implemented for real servers by configuring them with  source-nat  command:

server real r1 192.168.100.20
source-nat
server source-nat-ip 192.168.100.100 255.255.255.255 0.0.0.0 port-range 2 port-alloc-per-real 
server real r1 192.168.100.20
  source-nat

Per-real-server source NAT with ACL

Same as automap per-real-server, but in this case, SNAT is implemented only for traffic originating from private 192.168.100.0/22 network by utilizing an  access-list. This way, the access to the VIP from other real servers and the client requests from the Internet will not be subjected to SNAT, their IP will not change.

server source-nat-ip 192.168.100.100 255.255.255.255 0.0.0.0 port-range 2 port-alloc-per-real
access-list 1 permit 192.168.100.0 0.0.3.255 
access-list 1 deny any 

server real r1 192.168.100.20
source-nat access-list 1

 

HA considerations

If a SNAT configuration is used in an HA config, add the source-nat-ip into the vip-group in order for the secondary to take over the SNAT IP, in case of a failover:

server vip-group 1
source-nat-ip 192.168.100.100

Server Load Balancing : Source NAT – http://www.brocade.com/downloads/documents/html_product_manuals/VADX_03000_SLB/wwhelp/wwhimpl/common/html/wwhelp.htm#context=Virtual_ADX_0300_SLBGuide&file=slb_V_ADX.04.06.html

Brocade ADX – Fall Back Server

TERMINOLOGY:
  • Primary – A primary server is used by the ServerIron ADX when load balancing client requests for an application.
  • Backup – A backup server is used by the ServerIron ADX only if all the primary servers are unavailable for the requested application.
  • Local – A local server is one that is connected to the ServerIron ADX at Layer 2. The ServerIron ADX uses local servers for regular load balancing.
  • Remote – A remote server is one that is connected to the ServerIron ADX through one or more router hops. The ServerIron ADX uses remote servers only if all the local servers are unavailable.
  • By default, Local Real Server is considered to be “Primary” and Remote Real Server is considered to be “Backup”.
  • Local Real Server > Dedicated Servers
  • Remote Real Server > Cloud Servers

 

LOCAL REAL SERVERS:

  • Server definition starts with “server real” for Local Real Servers
server real web1 192.168.10.33
 port http
 port http keepalive
 port http url "HEAD /"
 port http l4-check-only
!
server real web2 192.168.10.34
 port http
 port http keepalive
 port http url "HEAD /"
 port http l4-check-only
!

 

REMOTE REAL SERVERS:

There are 3 Remote Real Servers. These are the Cloud Servers that are accessible via the RackConnected ASA FW.

  • Server definition starts with “server remote-name” for Remote Real Servers
  • source-nat is used for the Remote Real Servers
  • If the number of connections to the Remote Real Servers are expected to be more than 65K, it is better to use separate SNAT IP. Use the “Brocade ADX Source NAT” Axios documentation
server remote-name web3.domain.com 10.180.4.235
 source-nat
 port http
 port http keepalive
 port http url "HEAD /"
 port http l4-check-only

server remote-name web4.domain.com 10.180.5.109
 source-nat
 port http
 port http keepalive
 port http url "HEAD /"
 port http l4-check-only

server remote-name web5.domain.com 10.180.5.99
 source-nat
 port http
 port http keepalive
 port http url "HEAD /"
 port http l4-check-only
!

VIRTUAL SERVER

server virtual VS-5.5.5.5.5 192.168.99.30
 predictor least-conn
 port http sticky
 port http tcp-only
 port http lb-pri-servers
 port http reset-on-port-fail
 bind http web1 http web2 http
 bind http web5.domain.com http web4.domain.com http web3.domain.com http

By default, without “port http lb-pri-servers”, ALL the traffic will be sent ONLY to the “Local Real Servers”. Traffic will be sent to the “Remote Real Servers”, only if ALL the Local Real Servers fail. This is because the Local Real Server is considered to be “Primary Server” and Remote Real Server is considered to be “Backup Server”, by default.

port http lb-pri-servers

When we use the commands mentioned earlier ( port http lb-pri-servers ), all the real servers bound to the VS ( for port http ) will be considered to be “Primary” and traffic will be distributed across “Local” and “Remote” servers.

If we want one server to be “Backup” and all the other servers to be “Primary”, we would have to enter the command:

  • backup

under the relevant “Real Server” (Local or Remote).

CSW POLICY – CLIENT IP INSERTION

Create CSW Rule:

csw-rule "HOST_Domain" header "host" pattern "."

OR

csw-rule "HOST_Domain" header "host" exists

 

Create CSW Policy:

csw-policy "CSW_CLIENT_IP" 
match "HOST_Domain" forward 1
match "HOST_Domain" rewrite request-insert client-ip
default forward 1
default rewrite request-insert client-ip

We would have to use the separate “Match-Forward” rule followed by the “Default” rule since the “Default” rule on its own, without a “Match-Action” rule is not allowed by Brocade ADX.

Create Group ID for Real-Servers:

 port http group-id 1 1

Apply CSW Policy to Virtual Server:

 port http csw-policy "CSW_CLIENT_IP" 
 port http csw

Example:

server virtual VS-5.5.5.5 192.168.99.30
 predictor least-conn
 port http sticky
 port http tcp-only
 port http lb-pri-servers
 port http csw-policy "CSW_CLIENT_IP" 
 port http csw
 bind http web1 http web2 http
 bind http web5.domain.com http web4.domain.com http web3.domain.com http

Reference:

http://community.brocade.com/docs/DOC-1526/diff?secondVersionNumber=4

Brocade ADX Crash

Brocade ADX can crash due to failure of Management Processor (MP) or Barrel Processor (BP). In order to narrow down the issue, check the output for the following commands:Brocade ADX Crash

Brocade ADX can crash due to failure of Management Processor (MP) or Barrel Processor (BP). In order to narrow down the issue, check the output for the following commands:

#show version

This will show you the uptime of the MP

#sh ver | i uptime

The system uptime is 5 days 12 hours 44 minutes 17 seconds

Log into “rcon virtual” and execute the following command:

#asm show version

This will show you the uptime of the BP

#asm sh ver

Copyright (c) 1996-2009 Brocade Communications Systems, Inc.Boot  SW: Version 12.04.00 Nov 21 2011 15:09:57 PST label: dob12400Monitor  SW: Version 12.04.00 Nov 21 2011 15:09:57 PST label: dob12400System  SW: Version 12.04.00 Jul  9 2013 16:28:27 PDT label: ASB12400h The system has been up for 1 hours 8 minutes 35 seconds

As seen in the following outputs taken from an ADX that crashed, the uptime for the MP is more than the uptime for the BP. This would indicate that the BP reloaded while the MP did not reload.

Once this has been narrowed down, capture the dump file by running the following command:

#dm save mp (For MP)

#dm save bp 1 1 (For BP 1)

Brocade ADX – JSession Persistence

This is an example for JSession ID based persistence for Brocade ADX:


csw-rule "JSESSION" header "cookie" pattern "JSESSIONID=" case-insensitive
csw-rule "URI" url pattern "JSESSIONID=" case-insensitive                 
                                                                          
csw-policy "CSW_JSESSION" case-insensitive                            
 match "JSESSION" persist offset 0 length 32 passive-persist              
 match "URI" persist offset 0 length 32 passive-persist                   
 default forward 1

In the above policy, the persistence decision is made based on the first 33 characters (0-32) of the JSession ID. If this number has to be different, we would have to alter the CSW Policy to reflect the right number of characters. The csw-policy “CSW_JSESSION” would have to be utilized within the Virtual Server.

Brocade ADX – CSW HTTP Method

When using CSW Rule with POST method and load balancing based on incoming URI, you could run into a defect that prevents the right configuration. By default, “case-insensitive” CSW Rules would have to be added to “case-insensitive” CSW Policies. However, “case-insensitive” key word at the end of the CSW Rule for HTTP Method is NOT recognized by the Brocade ADX. Most browsers send the HTTP Method in Uppercase and hence, Brocade Engineers did not design the CSW Rule for HTTP Method to work with “case-insensitive” keyword.

Virtual ADX(config)#csw-rule “POST_Method” method eq “POST” case-insensitive

Virtual ADX(config)#csw-policy “CSW_POLICY” case-insensitive

Virtual ADX(config-CSW_POLICY)#match “POST_Method” forward 1

Rule “POST_Method” does not match policy “CSW_POLICY” case sensitivity type

What if you want to combine HTTP Method related CSW Rule (case-sensitive) with normal L7 load balancing rules based on case-insensitive URI ?

In this case, you can only create “case-sensitive” CSW Policy and not “case-insensitive” CSW Policy as the CSW Rule for Method doesn’t have a “case-insensitive” option. In effect, the URI that is utilized within the CSW Rule has to be “case-sensitive” and won’t match any case.

I wanted this configuration (case-insensitive):

csw-rule “POST_Method” method eq “POST” case-insensitive
csw-rule “URI_ADMIN” url pattern “/ADMIN/” case-insensitive
csw-rule “URI_STAGE” url pattern “/STAGE/” case-insensitive

csw-policy “CSW_POLICY” case-insensitive 
match “URI_ADMIN” forward 1026
match “URI_STAGE” forward 1026
match “POST_Method” forward 1027

Settled For This Configuration (case-sensitive):

csw-rule “POST_Method” method eq “POST”
csw-rule “URI_ADMIN” url pattern “/admin/” 
csw-rule “URI_STAGE” url pattern “/stage/” 

csw-policy “CSW_POLICY” 
match “URI_ADMIN” forward 1026
match “URI_STAGE” forward 1026
match “POST_Method” forward 1027

Issue seen: 12.4D code.

Brocade ADX – TCP Profile & Slowness

With the newer Brocade ADX code versions (12.4G+), it is possible to alter the TCP profile settings that is associated with the Virtual Server.

Why Change the TCP Profile:

I have run into issues when there is slowness in data transfer between the client and the Brocade ADX. Usually, I have seen this happen under 2 configuration setting:

  • CSW (Layer 7 rule is enabled)
  • SSL is terminated on the Brocade ADX

When we enable CSW or terminate SSL on the Brocade ADX, it tends to act as a “Full Proxy” device with TCP stack for Client-Side connection and another TCP stack for Server-Side connection. Somehow this setting will result in sub-optimal performance.

A sample TCP Profile:

tcp profile CLIENT_TCP
nagle off
delayed-ack off
rxbuf-size 524288
txbuf-size 2516544
tcp-wnd-scale 6

tcp profile SERVER_TCP
nagle off
rxbuf-size 2516544
txbuf-size 524288
tcp-wnd-scale 6

SSH@vADX(config)#server virtual VS-1.1.1.1

SSH@vADX(config-vs-VS-1.1.1.1)#port http tcp-proxy CLIENT_TCP SERVER_TCP

“CLIENT_TCP” profile changes the TCP profile setting that is client facing and the “SERVER_TCP” profile changes the TCP profile setting that is server facing. We can utilize the same TCP profile settings for both client and the server side, if required.

Ideally, I would recommend using 12.4U code version with the following TCP profile:

tcp profile TCP-PROFILE
delayed-ack off
rxbuf-size 1024000
txbuf-size 1024000
tcp-wnd-scale 4
tcp-timestamp on
tcp-sack on

The last 2 options for “timestamp” and “sack” are provided in 12.4U code version and are not available in earlier code versions.

The TCP profile can also be altered at the SSL profile level as noted here.

ssl profile star.domain.com
keypair-file star.domain.com-key16
certificate-file star.domain.com-crt16
cipher-suite all-cipher-suites
enable-certificate-chaining
tcp-profile TCP-PROFILE
session-cache off

In the above case, the TCP profile is added to the SSL profile and this will be attached to the Virtual Server. With newer code version, we can directly attach to the TCP profile to the Virtual Server’s port using the “tcp-proxy” keyword as illustrated earlier.

ADX – SSL Simple Health Check

By default, when we configure SSL Health Check for a Real-Server:

server real RS-web8 192.168.200.232
port ssl
port ssl keepalive
port ssl url “HEAD /”
port ssl status-code 200 200

The ADX will perform a simple SSL handshake. It doesn’t really check the content/status code. In order to force the ADX to check the content/status code, we would have to add this command globally:

(config)#no server use-simple-ssl-health-check

ADX Health Check

By default, when a real-server is created, the ADX will perform the following checks:

  • Layer 2 ARP
  • Layer 3 ICMP Ping

After the real-server is attached to the virtual-server using the bind command, the ADX will perform Layer 4 or Layer 7 checks based on the configuration.

It is essential to make sure that there is nothing blocking L2  or L3 checks between the ADX and the Server like a Firewall.

As there are multiple health checks for Brocade ADX, it is better to limit these checks to specific health checks to prevent any issues.

For example, when a real-server is created and “port http” configured,

port http url “HEAD /” is automatically added. Using “port http l4-check-only” will limit the health-check to Layer 4 health check instead of the Layer 7 checks. By default, for L7 “HEAD /” will mark the real-server as UP, if the response status code is 200-299, 401. This can be limited by using the command:

port http status-code 200 200

The above command will mark the real-server’s HTTP port UP, only if the response status-code is 200.

It is also essential to remember that the ADX will perform the L4 & L7 checks only once – when the real-server is bound to the virtual-server. If you require periodic checks, you would have to utilize the following command:

port http keepalive

The above command tells the ADX to keep checking the real-server’s port periodically instead of checking it only once.

A simple real-server would look like this for L4 Health-Check:

server real-server RS1 10.10.10.1

port http

port http keepalive

port http l4-check-only

port http url “HEAD /”

Reference:

ADX Health Check

 

CSW PHPSESSID Persistence

csw-rule “CSW_RESPONSE_Cookie” response-header “Set-Cookie” pattern “PHPSESSID”
csw-rule “CSW_URI” url pattern “PHPSESSID”
csw-rule “CSW_PHPSESS” header “Cookie” pattern “PHPSESSID”

csw-policy “POLICY_PHPSESSID”
match “CSW_RESPONSE_Cookie” passive-persist offset 0 length 7
match “CSW_PHPSESS” persist offset 0 length 7 passive-persist
match “CSW_URI” persist offset 0 length 7 passive-persist

The above CSW Rule & Policy will provide persistence based on the PHPSESSID cookie that is presented by the server.