Deploying Cisco Collaboration Edge – Updated

Update

Collab Edge is now supported.  The official Mobile-Remote-Access-via-Expressway-Deployment-Guide is located here.

I’m updating this document to reflect changes made in Expressway-C/E 8.1 that make importing the certificates MUCH easier.

Introduction

This document explains how to deploy Collaboration Edge with on-prem presence (IM&P/CUP) on a non-redundant set of Expressway-E and C VMs.  Deploying with WebEx Messenger is not covered here, but the bulk of the configuration is the same as far as the Expressway piece.

The biggest challenge in the initial deployment was finding all of the necessary documentation!  Things you need to know like certificate chaining, or OpenSSL are in various docs.  I’ve linked all of the documents that I used and tried to summarize things to make it quicker to deploy.

What you’ll need to deploy Collaboration Edge today:

• CUCM 9.1(2)+
• IM&P (CUP) 9.1+
• VCS or “Expressway” X8.1.1+
• A Collaboration Edge enabled Jabber client:  Cisco Jabber for Windows 9.7+, Cisco Jabber for iOS 9.6, Jabber for Android 9.6+ or Cisco Jabber for MAC 9.6+
• Updated jabber-config.xml on CUCM with RemoteAccess turned on  (This is no longer required for Jabber 9.6+)
• Two certificates (one for VCSe another for VCSc) – either signed by your own CA (OpenSSL or similar) or  publically signed certs like GoDaddy, Verisign, etc.

A few notes about nomenclature:

Collaboration Edge is the architecture umbrella term for the VCS/Expressway edge proxy for CUCM-registered clients (Jabber  and TC7.0 TP units).  It’s commonly used to refer to the Jabber piece of it, but will support endpoints too.  (The DX650 will support Collaboration Edge in a future release of firmware.  Traditional IP phones will not be supported, they will use VPN Phone or CUBE lineside proxy.)

Mobile and Remote Access (MRA) is the term used in VCS/Expressway documentation for the VPN-less Jabber (and CUCM-registered TC7.0 endpoint) proxy feature.

Cisco Expressway-Edge is the same software as VCS-Expressway, just packaged for CUCM registered endpoints.  Expressway-Edge is a VCS-Expressway that is deployed as a Mobile and Remote Access proxy or for traversal calls for CUCM registered endpoints.  There is a license file actually changes the title to say “Expressway-E” when it is loaded.  In the rest of the document, I will refer to VCS-Expressway, VCS-E, Expressway-Edge, Expressway-E as Expressway-E since we are primarily talking about MRA.

Expressway-Core is the same story.  It is VCS-Control software deployed as an MRA proxy only with the Expressway-C license loaded.  We call it VCS-Control when it is licensed for device registration, non-traversal calls, FindMe, and other features like Lync interop.  For purposes of this document I’ll call it the Expressway-C below.

Customers with a valid UCSS contract for UCL-Enhanced, CUWL-STD, or CUWL-PRO are entitled to Expressway-Edge and Expressway-Core for free (for MRA) and their license will reflect the Expressway names.  Licenses are charged for the other VCS features mentioned above.  Licenses are required and have a cost for B2B/B2C (Jabber Guest) calls through Expressway-C/E  Each box requires one media session license to get a session through.

VCS-C and VCS-E can have the MRA features turned on and run on a pair and do both functions.  We are still awaiting clarification as to when you must break these apart and run a separate set of VCS (for B2B, interop) and Expressway (for MRA) servers.  (Update:  VCS is supported for limited sized deployments.)

Expressway licenses should be orderable via PUT, or you can use your existing VCS severs by upgrading to X8.1.  (Update: I posted a later post that discusses what to order.)

Prepare CUCM for MRA

1) Create an AXL user if you don’t already have one on CUCM and IM&P.  There’s a good guide here –http://www.uplinx.com/cleanuptool/userguide/index.htm#page=Enable_AXL_on_CUCM.htm

2) Decide if you want to deploy valid security certificates on CUCM, IM&P and CUC.  You will likely want to do this independent of Collaboration Edge as all of the Jabber clients are no longer trusting self-signed certificates.  By providing a publicly trusted cert, Jabber won’t throw Invalid Certificate errors as you log in.  Granted they are only shown once during the very first login if the user accepts them on each client.  If you do put certificates on those components I’d suggest getting them for a 5-year term so you aren’t dealing with it in a year when the certificates expire.

Directory Lookup Considerations

 

MRA only supports UDS as the directory lookup service.  If you are inside you can use LDAP (EDI/BDI), outside UDS.

Jabber-config.xml Update for 9.6 clients

Update:  This section is no longer required as current versions of the clients (Win 9.7, iOS 9.6.1, Android 9.6, OS X 9.6) to do MRA by default, negating the requirement to pull the jabber-config.xml file first (expect in the case of split internal/external domains).

CUCM UC Service Profiles

Make sure that you’ve configure CUCM UC Service Profiles (this should have been done as part of your initial IM&P/CUP deployment and won’t be covered here) and assigned them to the end users.

Deploy Expressway OVAs

For new installations, you’ll need to download and deploy OVAs – http://www.cisco.com/en/US/docs/voice_ip_comm/expressway/install_guide/Cisco-Expressway-Virtual-Machine-Install-Guide-X8-1.pdf – See my previous post about upgrading to X8.1 if you’ve already got VCS installed – here

Recall there is a single OVA that does both Expressway-E/C and VCS-E/C that you need to deploy — it’s just a matter of how you configure and license (request via PUT as mentioned above) it as to what it is called.  Download the OVA from Cisco here

You’ll deploy the Expressway-C on your internal network (presumably on the same VLAN as CUCM and other UC components)  e.g. 10.10.1.30

You’ll deploy Expressway-E one of two ways.  Either on a stick in your DMZ (perhaps 10.99.99.30), or two-legged with the external interface in the DMZ network (e.g. 10.99.99.30 – or on your public address space), and the internal interface on the internal network (presumably on the same VLAN as Expressway-C and other UC components – e.g. 10.10.1.31).

You’ll need to trunk your DMZ to your ESXi host if you haven’t, or figure out how to deal with getting the Expressway-E external  (LAN2) interface in the DMZ network.

Once the VM’s are deployed edit the Expressway-E VM settings to put LAN2 in the DMZ network (alternately you can put LAN2 on the inside and LAN1 in the DMZ).  If you don’t see options for the second NIC, or options for NAT, you are missing the Advanced Networking license.  You need this in order to have it two legged, or do NAT.

Firewall Configuration

I don’t know how long I wasted on my first install because I forgot to modify the ACL to include the additonal ports that MRA requires.  I just assumed that beause VCS was working for B2B that it would work for MRA.  Not the case!

You’ll need to configure NAT on your firewall from a public IP address outside your network to the DMZ address of Expressway-E, or do 1:1 public to your DMZ if you’ve deployed it with a public address.

Look at p.258 of the Expressway Admin Guide for a concise list of ports.

 

A couple notes about DNS records

You will need two DNS servers for MRA to function properly.  Jabber decides if it is inside the network or outside the network depending on what SRV records it can resolve.  Depending on what records it resolves it will either try to use MRA or it will directly connect to CUCM/IM&P.

Internal DNS Server

Create two A records:

• sjc-expressway-edge-01.domain.com A – (make this name whatever you want) Pointing to the INSIDE interface of Expressway-E for two-legged deployments, or pointing to the DMZ address if it’s on a stick.  The record is used by Expressway-C to lookup and validate the certificate against.  You will use this hostname anywhere you are asked for the expressway server’s name whe configuring the C server.
• sjc-expressway-core-01.domain.com A – (any name you want) Pointing to Expressway-C.

Create two SRV records:

• _cisco-uds._tcp.domain.com SRV 0 0 port 8443 – Pointing to CUCM.  (NOT IM&P!)
• _cuplogin._tcp.domain.com SRV 0 0 port 8443 – Pointing to IM&P  (TBD if this is really required for Jabber 9.6 with IM&P 9.1 – I don’t believe it actually is)

When you launch Jabber, if it can resolve these DNS records, it knows it’s inside and pulls the service profile directly from CUCM and logs in to IM&P and CUCM.

External DNS Server

Create one A record:

• sjc-expressway-edge-01.domain.com A – (any name you want) Pointing to the public address assigned (or NATted) to your Expressway-E.

Create one SRV record:

• _collab-edge._tls.domain.com SRV 0 0 8443 – Pointing to Expressway-E (in our case sjc-expressway-edge-01.domain.com)

Configure Expressway-Edge and Expressway-Core

Follow this chapter of the Expressway Admin Guide  – Mobile and Remote Access (feature preview) beginning at p.52 but stop half-way down p.56 (before the beginning of the Certificates section).

A couple notes:  I did not enable TLS verify mode on my CUCM and IM&P server definitions because just wanted to get it up and running.  I’m suggesting putting real certs on CUCM, IM&P, and CUC, and turning TLS verify on, but this can be done later.

The admin guide is located here (p.52-56):

http://www.cisco.com/en/US/docs/voice_ip_comm/expressway/admin_guide/Cisco-Expressway-Administrator-Guide-X8-1.pdf

Certificates

Valid CA-signed certificates are required to setup the traversal zone for MRA.  You can either get public ones, or sign your own with your own CA.  I’ve done it both ways.  The major reason for a valid trusted CA-signed certificate is to stop Jabber from throwing a certificate warning on the initial MRA login to Expressway-E itself.  I highly recommend deploying a publicly trusted CA signed certificate.

Update:  This is fixed in Expressway 8.1.1  Ignore this section below:

Deprecated instructions for VCS 7.x:   The best document out there is this WebEx enabled Telepresence VCS Config document that describes how to chain up the intermediate cert properly here – http://www.cisco.com/en/US/docs/telepresence/infrastructure/tms/config_guide/webex_enabled_telepresence/cts_webex_vcse_cert.html

When importing the CA trusted certs, the key is to make sure the intermediate cert appears in the CA trust list ABOVE the root cert.

 

Expressway 8.1 Certificates

You will need to get a specific type of certificate, the multi-SAN (subject alternative names) also called a UCC certificate). 

Expressway-C CSR will be generated with the IM&P, and CUCM SANs.

Expressway-E needs the server itself and domain only as a SAN.

See the Expressway Certificate Guide for detailed information.

For Expressway-E follow this basic flow:

1. Generate CSR
2. Add UC Domain (domain.com) and XMPP server information
3. Download the CSR
4. Upoad the CSR file to the CA to get the certificate signed
5. Get the signed server PEM and the root/intermediate chain PEM back from the CA.
6. Upload the signed server cert to Expressway-E under Maintenance | Security Certificates | Server Certificate
7. Break apart the CA-intermediate-root certificates into individual PEMs for import – See the WebEX instruction for VCS 8.1 to learn how to do this.
8. Import into the Trusted CA certificate list: the top-level cert (“CA”), then the root cert, then the intermediate cert found under Maintenance | Security Certificates | Trusted CA Certificates
9. Reboot to make them active.

Follow the Webex instructions to break apart the CA-intermediate and root PEM into individual certs using Windows so that you can import them into the CA trusted cert list properly.

Repeat this procedure for Expressway-C.

For the customers that I’ve worked with using GoDaddy certificates.  I’ve worked with four certificates – Go Daddy Class 2 CA; Go Daddy Root Certificate Authority – G2; Go Daddy Secure Certificate Authority – G2; the server certificate itself.

I used Chrome on Windows to export the three Go Daddy certificates individually to Base 64 .PEM and then loaded them into the Expressway-E/C trusted CA list.  This worked perfectly for me after loading and rebooting the servers.  The UC traversal zones came right up.

 

Sign your own using OpenSSL if you’d like

If you want to use OpenSSL to create your own CA cert and sign your CSR, it is actually easier than you’d think.

Start at the bottom of p.13 of this document –

Click to access Cisco-VCS-Certificate-Creation-and-Use-Deployment-Guide-X8-1.pdf

You’ll follow the procedure twice.  Once for Expressway-E and once for Expressway-C.  Take the CA root cert that you generated and import it into the trusted list on both boxes, and then import your signed server cert on the appropriate box.

Traversal Zone Configuration

Resume the configuration tasks in the Admin guide on p.58 making sure to put the proper settings for both Expressway-E and Expressway-C.

If your certificates are good, you will see the traversal zone go active on both servers under Status | Unified Communications.  If not, double-check your configuration settings, and double-check your certificates.

Troubleshooting Zone Configuration

If the zone won’t go active and you think it looks good, check the logs to see what is happening.  My initial attempt where the certificates were not chained properly showed a continuous loop of TLS failures.  When I had my Expressway-C pointing to the external public address instead of the inside interface of Expressway-E, TLS looked good and even the SSH tunnel showed “up” but traffic wasn’t actually flowing.

The best place I found to troubleshoot this stuff was by putting the Expressway-C and E in “Devel mode” to enable the Experimental menu.  (Instructions for this are found on p.207 of the admin guide.)  The reason for this is because the CollabEdge/MRA feature is still considered experimental.  You need to look at the Developer Logs.  You can enable them for debug level as well as collect a tcpdump.

HTTP Whitelisting

 

Make sure to add your Unity Connection, and any other servers that Jabber needs access to.   Unity Connection requires it for Visual Voicemail to work.

Launch Jabber 9.6 internally

Update:  No longer required unless you are doing separate internal and external domains. (I’ll detail this in a later post.)

Launch Jabber on your client device on the inside network (so that it has direct access to CUCM/IM&P).  When you enter your email address Jabber should automatically discover your servers (using the before setup internal DNS SRV records).  If Jabber does not auto-discover, troubleshoot your SRV records.  The easiest method is to use dig or nslookup.

The quick nslookup method is to:

1. Launch the program,
2. Make sure it shows your internal DNS servers (that your device should be pulling via DHCP scope options)
3. Enter set type=SRV, then type _cisco-uds._tcp.domain.com.  This should resolve to the hostname of CUCM, or the IP address of it.
4. If using the hostname, exit nslookup and try to ping the hostname.

Once you enter your credentials you will likely be presented with several invalid certs to accept, and your client should connect and have IM, Presence, CUCM, CUC, and be able to IM and do voice/video calls.

Sign out and close Jabber

Launch Jabber 9.6 externally

 

Disconnect from your internal network and make sure your device is outside your network where a) it cannot resolve the internal SRV records, and b) it can resolve the external _collab-edge SRV record and access your Expressway-E from the outside.

Launch Jabber on your device.  Jabber will attempt to resolve _cisco-uds._tcp.domain.com and will fail to do so.  It will also attempt to resolve _cuplogin._tcp.domain.com and will fail.  It will then attempt to resolve _collab-edge._tls.domain.com and get pointed to the public IP address of Expressway-E.

It will then connect to Expressway-E, and a if everything is configured properly it will login and you’ll show connected to IM, CUCM and CUC!

Notes about iOS

 

On iOS the timeout for attempts to login is MINUTES long.   Be very patient for it to either succeed or fail.  It can take a significant amount of time to login successfully on the 9.6 build.  9.6.1 is supposed to be much faster.

If your login fails, click the Send Error Report and email it to yourself.  Open the ZIP file and look through (going from bottom to top) to see where the errors are.  The logs will include more than just the current login attempt, so note the time when you are attempting to login and look at the timestamps in the log.  This is critical so that you aren’t troubleshooting an old login that isn’t relevant to your current problem.

From my experience:

• When I had firewall issues, I was seeing CONNECTION_TIMEOUT errors when trying to login via MRA, but not when I was inside.
• When I had neglected to enable RemoteAccess in jabber-config.xml I was seeing RemoteAccess Policy errors.

Summary

I’m impressed with the ability to finally be able to do voice/video calls from anywhere!  It’s about time.  Collab Edge is still considered a Feature Preview by Cisco and isn’t TAC supported yet.  Please send me questions that you have as you attempt to deploy it.

-Mike

• Collab Edge
• cucm
• Jabber
• MRA
• VCS