This isn’t a Cisco ISE bug but it could affect ISE deployments. A customer had recently deployed several Cisco 3850s with Multigigabit at their headquarters. Initially, the switches were deployed with IOS XE 3.7.5. We tested the Cisco ISE configuration on those switches (Monitor Mode) and everything worked properly. The customer was able to authenticate users and endpoints using 802.1x and MAB as well as profile connected devices.
The customer ran into an issue with the IOS XE 3.7.5 code that caused link flaps. Another engineer engaged TAC to troubleshoot. When they discovered it was a bug that only affected 3850s with Multigigabit, the recommended fix was to upgrade to Denali (16.3.x). One stack was upgraded to 16.3.5b in order to verify that the link flap issue was resolved. We needed to verify that all authentication functions continued to work properly because of the IOS upgrade.
It was discovered that 802.1x authentication was no longer working. Checking the ISE RADIUS live logs showed that only MAB was coming through. There was no 802.1x authentication request coming from the switch. The switch logs showed the following:
Jan 22 09:22:23 CST: %SESSION_MGR-5-FAIL:Switch 1 R0/0: smd: Authorization failed or unapplied for client (38C9.860E.E745) on Interface GigabitEthernet1/0/6 AuditSessionID 0A4A05640000060F1E764F7C
This was repeated every time the client disconnected and reconnected. A quick search in Bug Tracker found bug ID CSCvg07470. This bug mostly matched our issue even though MAB was working. The bug was logged against a 3850 running 16.3.3. We opened a TAC case to see if the customer is seeing the same bug in 16.3.5b. It was determined that the customer was in fact hitting that bug. Unfortunately, according to the TAC engineer, there is no current Denali release that corrects this issue.
Because of the link flap issue being resolved in 16.3.5b, the rest of the switches will need to receive the upgrade. The good news is that the customer is only running Cisco ISE in Monitor Mode so no endpoint/user access is affected. The bad news is that the roll out to Low Impact and Closed mode will be delayed until a fix for this new bug is put into place.
Update 2018/01/29: It turns out that the issue I was running into is a configuration issue. The TAC engineer double checked my configuration and found that I was missing “dot1x pae authenticator” from the port config. This was not a required command when the switch was running 3.7.5 and C3PL was enabled but it is required in Denali.
So there you go. Lesson learned. I’ve made sure my configuration templates were updated accordingly.
Very useful article, thanks for it!
Hi,
Thank you for the post but still have the issue with Fuji 16.8.1a even if the port is configured with “dot1x pae authenticator”
interface GigabitEthernet1/0/6
switchport mode access
switchport nonegotiate
switchport voice vlan 200
ip arp inspection limit rate 20
authentication host-mode multi-domain
authentication open
authentication order dot1x mab
authentication priority dot1x mab
authentication port-control auto
authentication periodic
authentication timer reauthenticate 65535
authentication violation protect
mab
dot1x pae authenticator
dot1x timeout quiet-period 5
dot1x timeout tx-period 3
storm-control broadcast level 1.00
storm-control action shutdown
spanning-tree portfast
spanning-tree bpduguard enable
spanning-tree guard root
ip dhcp snooping limit rate 10
I haven’t had a chance to play around with a switch running 16.8, yet. It looks like you are using the old style port configuration. Are you doing C3PL/ISBN 2.0 or ISBN 1.0? You could try the Denali C3PL configuration template I have listed here.
Hi Brad,
Thank you for your answer, I will try to select the commands that fit to my scenario.
The difference here is that I am using NPS, the authentication is working but what’s bother me is that the switch always send the same message (the same one in the post) either for successful authentication or failed one, note that the devices (mab/dot1x) authenticate without problem.
# sh auth sess int g1/0/3 details
Interface: GigabitEthernet1/0/3
IIF-ID: 0x1188C1CC
MAC Address: 2880.45d8.0e68
IPv6 Address: Unknown
IPv4 Address: 192.168.213.114
User-Name: domain\usertest
Status: Authorized
Domain: DATA
Oper host mode: multi-domain
Oper control dir: both
Session timeout: 65535s (local), Remaining: 51020s
Timeout action: Reauthenticate
Common Session ID: 0A0E640D000000E8B8BE0E9F
Acct Session ID: Unknown
Handle: 0x1f000053
Current Policy: POLICY_Gi1/0/3
Server Policies:
Vlan Group: Vlan: 213
Method status list:
Method State
dot1x Authc Success
—————————————-
Interface: GigabitEthernet1/0/3
IIF-ID: 0x1405DCC6
MAC Address: f8a5.15c0.271a
IPv6 Address: Unknown
IPv4 Address: 192.168.200.45
User-Name: f8a515c0271a
Status: Authorized
Domain: VOICE
Oper host mode: multi-domain
Oper control dir: both
Session timeout: 65535s (local), Remaining: 47177s
Timeout action: Reauthenticate
Common Session ID: 0A0E640D000000B2B49B40A1
Acct Session ID: Unknown
Handle: 0x6e000033
Current Policy: POLICY_Gi1/0/3
Server Policies:
Vlan Group: Vlan: 200
Method status list:
Method State
dot1x Stopped
mab Authc Success
I know this is an old post , but if someone reads it could you confirm. We are planning to upgrade our 3650 switches from 3.7.5E IOS to 16.9.5 . So we when do that does the ISE config on the ports get auto-upgraded or we need to remove the config , upgrade the switch to 16.9.5 and then add support ISE config.