My blog; no-one else's

Author: scotia (Page 1 of 2)

Juniper MX10003: Port speeds and ASICs

Each MX10003 comes with two MPC slots, each with two MIC slots. One slot is pre-populated with a fixed, 6-port PIC, while the other slot is free to accept a MIC. Keen-eyed readers will point out that a PIC goes into a FPC, and Juniper certainly don’t make life easy with their terminology. What the marketing team calls an MPC the Junos team calls an FPC; the same applying to MIC and PIC. These interpretations may not be perfect but will serve for this article. From here on I will be using the FPC/PIC nomenclature unless other terms better apply.

FPCs, PICs and ports use 0-based indexes, with FPCs numbered from bottom-up, PICs from right-to-left, and ports from top-to-bottom, left-to-right. The bottom-most FPC and right-most PIC provides ports (et|xe)-0/0/0 through 0/0/5 with ports numbered left-to-right. The top-most FPC and left-most PIC (when populated with a MIC) provides ports (et|xe)-1/1/0 through 1/1/11 with ports numbered left-to-right with odd numbers at the top, even at the bottom.

Each FPC provides three Eagle (EA) ASICs with ports from both PICs in the same FPC assigned to them. The diagram below shows the port assignments.

Each ASIC has four MACs* of which there are two types. Avoiding the terminology, let’s call these two types of MAC type-1 and type-2. Type-1 MACs support 100G-speed interfaces only, while type-2 MACs support all of the supported interface speeds: 100G, 40G, 4x10G and 4x1G, the latter two being channelised and requiring a fan-out cable to deliver each interface. The number of ports that each MAC type differs, too. Type-1 MACs support a single (100G) interface while type-2 MACs support a single 100G, or three 40G, or 12 10G, or 12 1G interfaces. The four MACs provided by each ASIC are in pairs: two type-1 MACs are available, as are two type-2 MACs.

So for each port group as shown above there are four MACs, two of type-1 and two of type-2. Interfaces are assigned to MACs in the following manner:

  1. The first two 100G interfaces are assigned to the two type-1 MACs
  2. Further 100G interfaces are assigned to the the type-2 MACs, one interface per MAC. Therefore if four 100G interfaces are configured in a port group no further interfaces are possible (all four MACs have been used)
  3. If any type-2 MACs are available each one can provide three 40G interfaces (or three 4x10G or three 4x1G).

One other consideration to this is the pic-mode. Each FPC operates in one of two modes: pic-mode and port-mode. I believe both PICs in an FPC must be in the same mode. In pic-mode, each port on a given PIC operates at the same speed and ports within the same group on different PICs can belong so the same MAC. In port-mode port speeds are independent, however ports on different PICs CANNOT belong to the same MAC.

MX10003 PFE Eagle ASIC MACs

In pic-mode the following configuration for a given port group is possible:

  1. PIC 0, ports 0 and 1 running at 40G (or 4×10 or 4×1). These consume two of the three 40G interfaces that one of the two type-2 MACs provides. There is now a single 40G port available on the first type-2 MAC, and three available on the second type-2 MAC.
  2. PIC 1, ports 0 – 3 running at 40G (or 4×10 or 4×1). Port 0 consumes the last interface on the first type-2 MAC, while port 1 -3 fully consume the second type-2 MAC.
  3. Both type-1 MACs are available and can be used for 0, 1 or 2 100G interfaces.

Note that the allocations above, PIC permitting, can be shuffled around.

In port-mode, however, the above configuration is not possible. This is due to the limitation that MACs cannot be shared between PICs in port-mode:

  1. PIC 0, ports 0 and 1 running at 40G. Two of the three 40G interfaces on the type-2 MAC are now consumed, the third being unavailable to PIC 1 (and therefore wasted). The remaining type-2 MAC can provide three 40G interfaces.
  2. PIC 1, ports 0 – 2 running at 40G. The second type-2 MAC is now fully used.
  3. Both type-1 MACs are available and can be used for 0, 1 or 2 100G interfaces.
  4. The sixth port cannot be used.

Juniper helpfully hides this complexity in a port checker tool ( but hopefully the information above helps inform what happens in the background. The information in this post is mostly from

*I’m not sure what this initialism is short for.

Cisco WLC, MAC authentication and Passthrough WebAuth

While Passthrough WebAuth upon MAC authentication failure is not supported on Cisco WLCs, it is easy to workaround this by doing the following.

The first step is to create a user to be used as a dummy account that the web-form will silently submit when the user agrees. This user should be created depending on where you authentication happens (radius or local).

The second step is to modify the passthrough HTML file (login.html, for example) to include a hidden <div> that contains the dummy user’s login details:

<div id="hiddenForm" style="display: none;">
<h4 align=center><font color="#000000">Username<input type="TEXT" name="username" SIZE="25" MAXLENGTH="63" VALUE="webauthdummy"></font></h4>
<h4 align=center><font color="#000000">Password<input type="Password" name="password" SIZE="25" MAXLENGTH="63" VALUE="webauthdummy"></font></h4>

Just make sure the above is within the <form> block. The above can be simplified; I got lazy and just modified Cisco’s webauth examples.

<h4 align=center><font size="5" color="#000000"><input type="button" name="Submit" value="ACCEPT" class="button" onclick="submitAction();"></h4></font><p></p>

As above I also modified the submit button slightly to say “ACCEPT” as the passthrough form does.

And that’s it. You can now set the “On MAC Filter failure” when using the Web Policy Layer 3 Security feature.

FreeRADIUS, MySQL and Cisco WLC AAA Override

Here’s how to get AAA Override working on a Cisco WLC using FreeRadius backed with MySQL:

  • Enable AAA Override on the WLAN on the WLC
  • Create a Dynamic Interface on the WLC with a vLAN and IP address
  • Add the following to the radreply MySQL table:
radreply table

The NAS (WLC) will receive these parameters in the Radius reply packet when user username authenticates and will place the user on the interface with the corresponding vLAN.

Cisco password type 7 Vigenère cipher seed/keyword

If you were ever wondering what the seed (keyword) is for Cisco’s password encryption type 7 (which uses the Vigenère cipher), in ASCII it’s:


or in hex:

0x64, 0x73, 0x66, 0x64, 0x3b, 0x6b, 0x66, 0x6f, 0x41, 0x2c, 0x2e,
0x69, 0x79, 0x65, 0x77, 0x72, 0x6b, 0x6c, 0x64, 0x4a, 0x4b, 0x44,
0x48, 0x53, 0x55, 0x42, 0x73, 0x67, 0x76, 0x63, 0x61, 0x36, 0x39,
0x38, 0x33, 0x34, 0x6e, 0x63, 0x78, 0x76, 0x39, 0x38, 0x37, 0x33,
0x32, 0x35, 0x34, 0x6b, 0x3b, 0x66, 0x67, 0x38, 0x37

Private vLANs

Having just implemented private vLANs on a Cisco 3750X switch I thought I’d share some findings.

A private vLAN configuration consists of a primary vLAN, zero or one isolated vLANs, and zero or more community vLANs.

When associated with each other, primary, isolated and community vLANs limit the switch’s forwarding scope for frames.

For a normal vLAN the forwarding scope comprises all ports to which that vLAN is assigned.

For a pvLAN the scope changes:

  • the primary vLAN is a normal vLAN with a normal scope; frames are forwarded to any or all ports to which the primary vLAN is assigned
  • an isolated vLAN is a point-to-multipoint topology, much like the Frame Relay NBMA, where the scope is limited to a single port to which the isolated vLAN is assigned and all associated primary vLAN ports
  • a community vLAN’s scope is a hybrid of that of primary and isolated vLANs; it comprises all ports to which the community vLAN is assigned and all associated primary vLAN ports

Treatment of broadcast and unknown destination frames received on a given port is as follows:

  • primary vLAN: frames are forwarded to:
    • all other ports in the primary vLAN, all ports in the isolated vLAN, and all ports in all community vLANs
  • isolated vLAN: frames are forward to:
    • all ports in the primary vLAN, while having their vLAN translated to the primary vLAN number
  • community vLANs: frames are forwarded to:
    • all ports in the same community vLAN and all ports in the primary vLAN, again having their vLAN translated to the primary vLAN number

Another way to look at private vLANs is that they provide vLAN translation on ports depending on which way frames are flowing:

PrimaryNo translationP->IP->C
IsolatedI->PNot permittedNot permitted
CommunityC->PNot permittedNo translation
pvLAN translation

You can see that ports, regardless of their assigned pvLAN type, receive frames with their own vLAN id.

An interesting consideration with pvLANs is with layer 3 SVIs and pvLAN trunks with a router connected. If there is no IP ACL on the SVI/router port denying such traffic, any host, regardless of the vLAN type assigned to its port, can reach any other host in the same private vLAN configuration, simply by creating a static ARP entry where the destination host’s IP is mapped to the SVI/router port’s MAC address.

This is because any host can reach a router on a primary vLAN (using its MAC address), which then is able to reach any other host within the pvLAN.

Using LetsEncrypt certificates with Cisco WLC WebAuth

Assuming you’ve got a nice fresh certificate from LetsEncrypt and are in the directory where it, your key, and the LE root certificate1 (TrustID X3) lives:

cat ca.cer dst.cer > fullchain-plus-root.cer
openssl pkcs12 -export -in fullchain-plus-root.cer -inkey -out domain-fullchain-key.p12 -clcerts -passout pass:wlc
openssl pkcs12 -in domain-fullchain-key.p12 -out domain-fullchain-key.pem -passin pass:wlc -passout pass:wlc
cp -a domain-fullchain-key.pem /tftpboot/certs/
transfer download mode tftp
transfer download datatype webauthcert
transfer download serverip 
transfer download path /certs/
transfer download filename domain-fullchain-key.pem
transfer download certpassword wlc
debug pm pki enable
transfer download start

Check the logs to make sure the import was successful. If so you will be reminded to reboot. If so:

reset system

Cisco ACLs and the ‘established’ keyword

While writing software to convert Cisco ACLs to VyOS’s firewall syntax I got to wondering what the ‘established’ keyword meant on TCP ACLs. Although my Internet facing ACL is connected to CBAC (and therefore stateful), my inter-vLAN ACLs are stateless, and so the ‘established’ keyword comes up a lot. Additionally, I have yet to successfully get the VyOS stateful firewall working so I needed to convert the Cisco ‘established’ to the VyOS equivalent.

I wrote a script that called nmap with all 64 combinations of TCP flags and send the traffic from a host to a destination in another vLAN across my IOS router. The destination ran tcpdump to collect the data. Firstly the script ran without any access-list to verify that the destination saw all 64 packets, and secondly with an access-list with the ‘established’ keyword set.

The results were:


Of 64 possible combinations 28 are permitted. Note that the second, third and fourth columns are the first column plus ‘PSH’, ‘URG’, and ‘PSH,URG’

The table can be summarised as:

  • ‘RST’ with all flags except ‘SYN’ (16 combinations)
  • ‘ACK’, ‘ACK,SYN’, ‘ACK,FIN’ with ‘PSH’,’URG’, and ‘PSH,URG’

Apple Watch and IPSEC/ESP

Something I noticed when my Apple Watch was associated to the wrong SSID (and was placed on a vLAN different to that of its paired iPhone) was the repeated attempts it was making to set up an IPSEC connection to the iPhone.

I saw this in the logs of the firewall that sits between the vLANs:

2019-10-29T16:56:20+11:00 kernel: [5242514.662174] [ALEXT-Zone-Media-340-R] IN=eth0.6 OUT=eth0 MAC=00:50:56:00:00:00:08:f4:ab:00:00:00:08:00 SRC= DST= LEN=120 TOS=0x00 PREC=0x00 TT
 L=63 ID=64399 PROTO=ESP SPI=0xabd8402

Here the Watch is and the iPhone

Apple’s iOS Security Guide does not mention an IPSEC connection specifically but does discuss secure WiFi connections in case Bluetooth is unavailable.

avahi-daemon and incrementing Apple device names

For a long time on my network I noticed my various Apple devices incrementing their hostname (specifically the name advertised with mDNS/mdns-sd – as appears in the ‘Sharing’ section of ‘System Preferences’). I lazily put this down to a poorly configured (by me) WLC and/or avahi-daemon running on my FreeBSD NAS and moved on to bigger problems.

My recent purchase of an AppleTV and it’s placement on a different subnet to my wireless clients (iPhones, etc.) meant getting cross-subnet mDNS working, and so I started playing with the WLC, the Avahi daemon, as well the mDNS service routing on my Cisco 1941.

Hoping the Cisco IOS router would solve everything I removed all of the interesting mDNS configuration from the WLC, stopped the avahi-daemon, and concentrated on getting the router mDNS service working. After much Googling and perusal of configuration guides and CiscoLive! PDFs I applied what I thought was an appropriate configuration. Unfortunately the implementation proved to be quite buggy (although it appeared to have potential). I may revisit it with a newer IOS and some spare time.

Something common to Cisco’s IOS implementation of an mDNS cache and avahi-daemon (and possibly mDNS on the WLC) is that they can act as mDNS advertisement caches, responding on behalf of hosts on subnets other than that on which the request arrived (I assume they ignore requests seen on the same interface as the searched-for service/hosts). Cisco also mention that cache interaction with Apple’s sleep proxies can behave badly and helpfully describe how to block the sleep proxy mDNS service advertisements into the cache.

Another common feature is mDNS relaying across subnets, whereby advertisements and requests are simply forwarded across participating interface. In a CiscoLive! presentation Cisco warns against implementing this – I guess to avoid good-old broadcast loops should a second relay be present. Who knows whether IP TTL would even help here).

Back to the incrementing hostnames. While watching avahi-browse -a, tcpdump, and repeated restarts of the AppleTV I noticed a reliable +1 to the AppleTV Airpay name (per restart). Interestingly was that avahi-browse was not showing any withdrawal of the AppleTV’s services as it shutdown. So when it came back up it checked whether any other device was using the proposed name, to which request avahi-daemon helpfully replied. Of course, the AppleTV had to choose a different name, and incremented it by one.

The answer was to disable the cache, effectively limited the cache’s size to zero. The line “cache-entries-max=0” in the configuration file and a restart was all that was needed. Turning on the relaying with “enable-reflector=yes” and I was off the races.

A couple of notes:
the avahi-browse command only talks to the avahi-daemon, and only via DBUS, so if the daemon has yet to start, or it’s configured with “enable-dbus=no” avahi-browse will be useless
all of my devices talk happily to each other, although I had a couple of quirks. 1) I could not get my Mac and the AppleTV to pair using Apple Configurator 2 – I would be prompted with a password to enter into the Mac and once entered the AppleTV would say it was paired, but the Mac would time out with a “pairing failed (49)” error. Though it irked me so, I gave up and associated my Mac with the SSID on the same vLAN as the AppleTV and instantly they paired. Upon moving my Mac back to original vLAN the pairing remained. 2) sometimes the remote function on my iPhoneXS would allow gestures to control the AppleTV, but none of the buttons worked – sometimes that behaviour was reversed. I have no idea what caused/fixed it.

AppleTV 4K Airplay across subnets

After a protracted battle with mDNS across broadcast domains (which I have yet to satisfactorily solve) I was at a point where I could see my AppleTV on my iPhone as an AirPlay target.

However when selecting the AppleTV in The Music or PocketCasts apps playback immediately stopped, and the tick would disappear from the GUI.

Knowing it was probably access list related, I turned on logging of the deny clause of my access list to discover that there were drops of UDP/319 and UDP/320. Strangely these well-know ports in /etc/services relate to PTP, the precision time-keeping protocol. Once I opened up these two ports Airplay started working.

As a side note, playing movies from the Photos app to the AppleTV worked before I added those ports. I think this is because that stream uses TCP, which was already permitted on my media subnet (where the AppleTV resides) for established sessions.

« Older posts

© 2022 Scott's Blog

Theme by Anders NorenUp ↑