ARP is not working on Cisco ASR 1001 X

Problem: Cisco ASR router is loosing connectivity to its directly attached Ethernet neighbors. In this situation interface status is still up, packets are going in and out on both ends, even IPv6 was still working. The actual problem was that the Cisco ASR was ignoring all ARP responses from its neighbors and the ARP table to this interface was empty. Later the same happened on a second interface.

A temporary work around was to reboot the router.

Solution: Cisco support suggested a software upgrade, even though the software was only some weeks old. After the software upgrade the error didn’t happen again until now.
The old IOS version was: asr1001x-universalk9.03.16.03.S.155-3.S3-ext.SPA.bin
The new IOS version is: asr1001x-universalk9.03.16.04a.S.155-3.S4a-ext.SPA.bin

The only fix that possibly fits to the problem is:

https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20160804-wedge

“A remote attacker can cause an interface wedge and an eventual denial of service condition”

What’s an “interface wedge”. Cisco bug reports were more precise years ago.

 

Update Cisco Catalyst Software

I had to update the software of a new Cisco Catalyst 4948 yesterday.
As usual I did:

copy tftp://<hostname>/<filename> bootflash:
conf t
boot system flash bootflash:<filename>
exit
reload

But the switch ignored the new software image.
During boot it said:

Booting first image from bootflash

Solution: The config-register was set to 0x2101 right out of the box. I had to change this to 0x2102. With this value the switch honors the bootvar and boots into the configured system image.

conf t
config-register 0x2102
exit
reload

The config-register has different meanings on different cisco plattforms.
For the Catalyst 4948 the config-register contains a bit field explained at:

http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/31sga/configuration/guide/supcfg.html#wp1051990

Windows Network Load Balancing NLB and Cisco Routers/Switches

Problem: Windows NLB IPs are are not reachable through and from Cisco routers and switches. NLB services could be IIS arrays, Exchange CAS arrays, etc.

Solution: NLB mode was set to Multicast. In this mode Windows incorrectly uses multicast mac addresses. Set the NLB mode to Unicast and configure static mac address table entries on your switch to prevent broadcast flooding.

Discussion: Windows NLB does only work if all members of an array get all packets for a balanced service. To achieve this, Windows knows 3 modes of load balancing: Unicast, Multicast, and Multicast IGMP. And all of them have problems:

Unicast: Windows uses a normal mac address for his virtual IP address, but never sends any packet from this mac. The switch never learns a mac address entry and has to broadcast all packets for this mac. (Broadcast Flooding)

Multicast (IGMP): Windows uses mutlicast macs for the virtual IP. This seams correct, because this way the switch could learn which ports are part of the array and which are not (IGMP), but the problem is: an arp request for an unicast address (virtual IP) must not resolve to an multicast mac address. Cisco switches simply ignore this arp responses. Multicast mac addresses start with the lsb bit of the first bit set, typically 01:XX:XX:XX:XX:XX or 03:XX:XX:XX:XX:XX.

My solution was to use Unicast mode and don’t use IGMP. The other solution would be to statically set arp and the mac-address-table on the cisco switch, and force it to use the incorrect mac address.

BTW: A Network Load Balancing mechanism where every array member receives all traffic, in no real “Network” load balancing, because you dont’t reduce the traffic per server, it just adds additional computers and no additional network capacity.

More info: http://www.cisco.com/en/US/products/hw/switches/ps708/products_configuration_example09186a0080a07203.shtml

Cisco has weird naming conventions for groups of interfaces

If you want to bind some ethernet interfaces into one logical interface between two ciscos switches you have to configure an interface first:

interface Port-channel 1

then add some interfaces into this new interface:

interface GigabitEthernet 1/0/1
        channel-group 1 mode active

and you can check the status with:

show etherchannel 1 summary

Three names for the same thing. Does anybody know, why ?

 

L2TP tunnel between two cisco routers

Problem: a Cisco 1941 [15.1(2)T2] (Client) tries to connect to an Cisco7300 [12.3(22)] (Server) using L2TP, but after successful authentication of the client the server drops the connection.

Solution: The client tried to authenticate the server, but the server had no password for authentication. Normally only the client authenticates to the server in ISP environments. Server sends “O CHALLENGE” client sends “I RESPONSE”. In this case the Client sent “O CHALLENGE” back. I debugged this problem by removing the following line from the clients interface Virtual-PPP section:

ppp authentication chap pap callin optional

Usually this means “authenticate a client (optionally) using chap or pap when it calls in”, but on this router/ios-release the client tried to authenticate the server when this line is present.

Cisco Access Point blocking PPTP

Vienna: 10. August 2011

Problem: A network of some Cisco Access Points AIR-AP1142N-E-K9 connected to an small router works good as expected but when the user tries to connect to en PPTP VPN the connection fails with a time-out.

Solution: it was no NAT problem in this case (I fixed this before). Cisco IOS 12.4(21)?? has officially a bug not sending GRE (Proto:47) packets from the network to the client. IOS Version 12.4(25d)JA fixed this bug.

Strange! An access point in bridging mode filters depending on protocol numbers. So I guess it’s not really a bridge that Cisco implemented in this IOS device.