Notes about Upgrading to CSR 11.5

Helped a customer upgrade from 11.0 to CSR 11.5, CUCM 11.5(1)SU1; IM&P 11.5(1)SU1; CUC 11.5(1)SU1.

Unity Connection 11.5

You must apply ciscocm.cuc_11.5SU1_pre_upgrade.cop.sgn before you upgrade to 11.5 because of bugid CSCvb02774.  The install of the patch is straightfoward and does not require a reboot.  I also ran a utils iothrottle disable to make the upgrade run faster (since it was being done after hours.)

If you’re upgrading from 10.x or earlier it is CRITICAL to increase your VM RAM to 6GB.  (This was something I ran into when going to 11.0.  If you leave it at 4GB  it will not function properly at all.)

The upgrade ran normally and took a quite a while for the switch-version to complete.

On a site note, I noticed that the new Unity Connection (CUC) 11.5 .ova files define a 200GB HDD for the bigger VM.  I investigated increasing my HDD from 160 to 200GB, but found out that CUC does NOT support dynamic resize of the HDD.  This will cause the partition to be unaligned and you’ll get to rebuild CUC from scratch.  So leave it at it’s current size.

CUCM 11.5

To save time during the upgrade window, the day before I preloaded the 11.5 ISO on my remote ESXi datastores so that it wouldn’t take forever for the ISO to SFTP over to the remote offices (they have limited bandwidth) , then I attached those ISOs as virtual DVDs to the CUCM servers via vShpere and then launched those upgrades as though they were coming from DVD instead of a remote file server.

The first attempt to launch the upgrade on the Pub failed with the old “common parition doesn’t have enough space” business.  I used RTMT to decrease the Low and High logging watermark to 45 and 40 respectively (and restarted the log partition monitoring service) to create room.

Purge Log Files by Changing the Log Partition Watermarks

• Another way to create additional disk space is by changing the high and low watermarks on the system. This informs Unified CM of the numbers of log files to purge once the watermark is reached. Use RTMT as follows:

1. Launch RTMT and log in to the desired cluster.
2. From the left pane, select Alert Central.
3. On the right pane, double-click LogPartitionHighWaterMarkExceeded. Change the threshold value to 40.
4. On the right pane, double-click LogPartitionLowWaterMarkExceeded. Change the threshold value to 45.
5. This data is polled every five minutes. Allow five to 10 minutes and then check the drive partitions for additional disk space by using one of the methods described above.

http://docwiki.cisco.com/wiki/Unified_CM_L2_Upgrade_Disk_Space_issues#Purge_Log_Files_by_Changing_the_Log_Partition_Watermarks

 

As usual, I ran the Pub first (without switching version), when it completed, I ran the Subs (also without switching versions).

If you’re coming from 11.0, the utils iothrottle disable command is not necessary.  (You can try to run it but CUCM 11.0 tells you it is unneeded.)

I rebooted the Pub and then Subs as normal.

IM&P 11.5

This was also a typical upgrade.  The switch-version took a LONG time for services to come up on the reboot.

 

 

CE 8.2 code on the DX70 and DX80

***UPDATED***  The Devpack, CE8.2.1 and conversion cop are now available on CCO.

CE 8.2.0 firmware for the DX70 and DX80 posted to CCO today.  (But of course we’re still waiting for the Devpack with the QED that will be released in the next week or so.)  The release notes are here.  The Official Conversion Guide is here.  Any instructions in this guide would override what I’ve said.

I’ve been running the beta for the last few weeks and can absolutely say CE code on the DX is MUCH more responsive, stable and usable than the Android-based code.  (That said the last few builds of 10.2(5) android code have been pretty decent.)

CE 8.2 code is absolutely the code to move to if your primary use for the DX is for video calls.

 

Why move to CE on the DX?

• Responsiveness.  No more lag!  It’s snappy.
• Stability.  Seldom, I had the random crash and full-reboot during video calls with Android.  While it was pretty rare, it was super frustrating.
• Video-centric user interface.  It run CE codec code now and feels like a Cisco codec (like SX10, 20, etc.) now.  It’s all about the video call.
• Registration to VCS/Expressway
• Far-end Camera Control
• OBTP (one button to press) meeting launch and TMS management
• Fully customizable wallpaper

Why stay on Android?

• You need Android applications.  CE doesn’t run any apps, period.
• You use the built-in Cisco Webex and Jabber Android apps on the DX.
• You need a local web browser.  CE doesn’t have a web browser built in.
• You are currently using Wi-Fi or Bluetooth.  The CE code doesn’t yet support Wi-Fi or BT.  (That’s coming in a follow-on release.)
• You need telephony features on the phone like CFwdAll, Shared Lines, Voicemail button, Auto answer.

What happens to Android?

Android will be supported on the DX70 and Dx80 for the life of the product.  Keep running it if you need features that will always only be specific to Android (local web-browser, Android Apps).  The DX650 will remain Android-only.

Converting to CE Code using CUCM

Note: You have a couple options to convert to CE code.   CUCM of course or, as the conversion guide notes, there is a public TFTP server on the internet provided by Cisco to convert a DX using.  As far as CUCM, you can convert the DX either onnet or connected via MRA registration through Expressway.

1. Upgrade to the latest build of Android code – 10.2(5)207 by installing the COP file on CUCM, restarting TFTP and rebooting your DXes.  (Either way you want to go to this code because of all of the bugfixes.)
2. Install the latest (Early July 2016) Devpack:
   CUCM 11.0.1:  cmterm-devicepack-11.0.1.22048-1.cop.sgn
   CUCM 10.5.2: cmterm-devicepack10.5.2.14076-1.cop.sgn
   CUCM 9.1.2: cmterm-devicepack9.1.2.16137-1.cop.sgn
3. Install the Devpack on CUCM as well as cmterm-synergy-ce8_2_1_no_defaults.cop.sgn (or latest version) so that it gets the Telepresence DX70 and Telepresence DX80 device type QED installed.  Reboot your cluster.  (Unless you’re on CUCM 11.5 which doesn’t need the immediate reboot!!)  Or bug someone you know for the standalone CE 8.2 QED in the meantime.
4. This devpack should have the CE 8.2 firmware, but if not install the CE 8.2 COP file – mterm-s52040ce8_2_1.k3.cop.sgn (or current); restart TFTP.
5. In CUCM, change the Phone Load of the existing DX80 to the CE 8.2 phone load name specified in the conversion guide.  For 8.2.0 it is sipdx80.ce821.rel.loads
6. The DX80 will take a few minutes (10-15) to upgrade to CE 8.2.1.
7. Take note of the MAC address of the DX80 in the CUCM device, because you are about to DELETE the DX80 device!
8. Delete the DX80 device from CUCM.
9. Create a new Telepresence DX80 device in CUCM and paste in the MAC address of the DX80 you just deleted.  Set the appropriate device settings and add an extension/SIP URI to the device.
10. On the DX80 itself, run through the startup wizard and pick UCM registration or UCM through Expressway (if your endpoint is registering through Expressway).
11. You’ve now got a DX80 on CE 8.2 code
12. Enabled Web Access in CUCM device settings so that you can get to the DX80 GUI.
13. Login to the GUI and set the admin password.  (This step may not be needed, setting the admin username/password was not available in earlier CE betas via the CUCM device setting page.)

 

Other Notes

• The Touch 10 doesn’t work on the DX80 or Dx70, you must use the built-in touch screen on the DX.

8800 Series 11.5(1) Firmware – Enhanced Line Mode

11.5 firmware for the 8800 series phones is on CCO now.

It comes with a bunch of cool new features.

Enhanced Line Mode

The coolest and most useful that I’ve seen requests for is the new Enhanced Line Mode.  (I’ll abbreviate it ELM even if there’s overlap with the PLM/ELM acronym.)  ELM allows all 10 buttons on the phones to be used a programmable line keys (PLKs).

The mode we’re used to includes 5 PLKs on the left, and 5 context-sensitive function keys on the right.  I like this mode, having gotten used to it back in the day with the 9900 series phones, but hear customers who need more than 5 PLKs (in particular for admin/receptionists who want more than 5 BLFs or Shared lines).

 

As you can see in the picture, I can now use all 10 buttons.

While the firmware is out now, there is a Devpack required to enable the ELM feature on the device configuration page.

The release notes indicate that you should get the latest Devpack from CCO, install it and reboot the cluster to enable the ELM setting.  The challenge you’ll hit is that the latest Devpack on CCO as of today (mid-June) doesn’t actually include the QED file that enables the ELM setting.  The Devpack that inclues the QED will release in the next couple weeks.

Look for a Devpack with a late-June/early-July date stamp if you want to turn this feature on.

Enhanced Do Not Disturb

The DND function has been updated to be much more obvious which is nice.

Other features to mention:

• Wi-Fi Security Enhancements
• Customized Dial Tone for SIP Phones

See the release notes for more information about the last two.

 

 

CUCM 11.5 Released – Warning!

CSR (CUCM/IM&P/CUC) 11.5 has been released!  It includes a lot of cool new features but there’s a warning that needs to be shared for customers who have legacy phones.

New Features of note:

CUCM 11.5

• “Hitless” Device Pack Installation.  Add new device types without an immediate reboot.  TFTP restart required, and Publisher reboot at next maintenance window.
• Read-only AXL user role.  (Finally!)
• PIN Sync to Unity Connection
• User-customizable  Display Name field searchable in the directory for users who want a nickname or short name.  (e.g. Mike White vs Michael White — both searchable)
• Directory search for MRA clients/devices
• UCM Scale Increase with same OVA spec
• Single SAML SSO Agreement per Cluster
• Next Gen Encryption (DOD Suite B)
• VMWare ESXi 6.0 support, Single SAML IdP per Cluster

 

IM&P (Jabber Server) 11.5

• Multi-device Messaging.  Finally copies of chats messages are sent to all clients, not the most recently active.  Read notifications sync’d across all devices.
• Persistent Chat HA and support for MS SQL (in addition to currently-supported Postgres and Oracle) for Persistent Chat functions.

Unity Connection

• Full video messaging.  In addition to video greetings, video messages can be left in inboxes.  Video playback is supported from the handset, and not Jabber at this time.  (This functionality requires Mediasence as the video engine.)
• PIN Synchronization between CUC and CUCM
• New Media Player replaces the old MediaMaster applet
• Support for Exchange 2016/Outlook 2016, Windows 10, MS Edge Browser
• Next Gen Encryption (DOD Suite B)
• VMWare ESXi 6.0 support, Single SAML IdP per Cluster

 

The warning is located in the release notes.  Certain legacy (i.e. REALLY OLD 12+ year EoS models) phones WILL NOT WORK with CUCM 11.5.

Update: A Field Notice has been posted.

This is a drastic change from previous versions were the phones are no longer supported and may not work properly anymore rather than disabling these models.  For example, I have an old 12SP+ at the office and it would register to CUCM 10, but not really work right.  In the case of 11.5, IT WILL NOT WORK.

Realistically the only models that I still see customers with are the 7935 Conference Station, 7920 Wireless, and 7910.

If you have these endpoints, understand that most have been EoS for over 12 years, and that there is a very aggressive phone trade-in program to get 20-30% extra discount (on top of your normal discount) to go to the new 8800 series phones.

Deprecated Endpoints

As of Cisco Unified Communications Manager Firmware Release 11.5, the following phones are not supported:•    Cisco IP Phone 12 SP+ and related models
•    Cisco IP Phone 30 VIP and related models
•    Cisco Unified IP Phone 7902
•    Cisco Unified IP Phone 7905
•    Cisco Unified IP Phone 7910
•    Cisco Unified IP Phone 7910SW
•    Cisco Unified IP Phone 7912
•    Cisco Unified Wireless IP Phone 7920
•    Cisco Unified IP Conference Station 7935

If you use any of these phone models on an older release of Cisco Unified Communications Manager and you upgrade to Release 11.5, the phone will not work after the upgrade completes.

Here’s an example from my CUCM where I have a 7905 defined.  The phones stays there, but this giant warning appears at the top.

Adventures in Upgrading to CSR 11.0

Now that all of the core CSR 11 components have had a service release under their collective belts, it’s go-time.  I helped a customer upgrade CUCM, IM&P and CUC from 10.5 to release 11.0.

• CUCM/CUC 11.0(1a)SU1
• CUCM IM&P 11.0(1)SU1
• Jabber for Mac and Windows 11.5
• Latest DX-series/8800 firmware
• Expressway C/E X8.7.1
• CWMS 2.6 MR1 Patch 1
• Security COP to address CSCuy07473 for CUCM 11.0(1)
• Permanent Licensing Surprises
• Holding UCCX at 10.6 for now…. (Agent/Supervisor issues)

 

CUCM 11.0(1a)SU1

This is a four node system (pub and 3 subs) running the latest 10.5 SU.  Upgraded the pub during evening hours and told it not to reboot the night before.  Once it had completed the upgrade, I ran the upgrade on the three subs and told them not to reboot.  The maintenance window was the next evening, so we didn’t make any changes during that window.

When trying to reboot the pub to the new version from the GUI it got into an ugly loop.  Switch version reported that an upgrade was still in progress.  Went to the upgrade menu option and it indicated that I had to assume control over the upgrade.  Did so and the log file showed that the upgrade had completed successfully and that the lock files were released.  Went back to switch version and it still indicated that it was in an upgrade…

Bailed on the GUI and issued the version switch from the CLI.  It didn’t complain at all and did the version switch from 10.5 to 11.  It was a faster process than I imagined, taking less than 10 minutes to reboot on 11.  I was a bit concerned if it was going to work given the GUI seeming to be in a loop, but it rebooted just fine.

I rebooted the subs all from the CLI since I didn’t (and now perhaps don’t trust) the GUI switch version.  They all rebooted quickly too and were up and running on v11.0

Important!  Please note that the OVA release notes indicate that the RAM should be upgraded to 6GB for the CUCM and CUC VMs – http://www.cisco.com/web/software/283088407/126036/cucm-11.0.ova.readme.txt.  This was pointed out by a kind reader, which I hadn’t noticed until my CUC was falling apart after the upgrade.  Moving this VM to 6GB was an immediate fix.

CUC 11.0(1a)SU1

This was a textbook upgrade that evening.  I’d prestaged 11.0 like CUCM and the reboot took about 15 minutes.  All was well until I applied the permanent licensing.  Which I’ll cover later

IM&P 11.0(1)SU1

I wasn’t able to pre-stage this upgrade so ran it the evening of the maintenance window.  It’s a fairly small system so it took about an hour to upgrade and reboot.  The reboot seemed excessively long and I was worried, but it came back and workstation Jabber  clients automatically connected.

In conjunction with this upgrade I updated my jabber-update.xml file and push out the latest 11.5(2) version of the Mac and Windows clients.  We also updated all of the user photos on the webserver that houses them to current pictures.  Jabber was hit and miss about actually pulling the new picture.  It seemed that you had to manually view the profile on some users to get it to pull the new picture.

Latest DX/8800 series Firmware

The DX-series firmware has been a bumpy bumpy bumpy road.  It’s finally pretty stable as of 10.2(5)154.  A newer 10.2(5)195 is out so I pushed that out as it has a number of bugfixes.  I also updated the photo location for the DX-series phones and they all now pull the photos correctly from the webserver that houses them.  The super secret URL to put in the Company Photo Directory is this:  http://<webserver ip address>/%%uid%%.jpg

I migrated the DX-es from Anyconnect VPN over to MRA through Expressway that night since this latest ASA Sev 10 Bugfix upgrade has caused an odd cert issue for the DX (not not normal Anyconnect software clients on other platforms).  Remote phone control does work properly from Jabber (that is VPNed in) to the phone that is connected via MRA.

CUCM 11.0 default firmware also had older firmware for the 8800 series phones so I pushed the latest 11.0 version and am anxiously awaiting 11.5 for some really cool upcoming features for the 8800 series.

Expressway C/E X8.7.1

Textbook upgrade.  I love the software that came from Tandberg.

CWMS 2.6MR1 Patch 1

This is still my favorite app to upgrade by miles.  Attach the ISO to the Admin VM in vCenter and press go from the GUI.  An hour or so later after a couple reboots of all the various VMs (Admin, Media, IRP) you kick it back out of maintenance mode and you’re done.

Security COP to address CSCuy07473 for CUCM 11.0(1)

This patch JUST released with the latest security fixes for CiscoSSL (a ciscoized variant of OpenSSL).  Install on each CUCM node and you’re done.  No reboot required.

Permanent Licensing

After upgrading everything to 11.0 everything kicked into 60-day temp license mode as expected.  (Upgrading to CSR 10.5 was bad news when it didn’t do what it was supposed to and CCX ate all of it’s licenses resulting in a P1 case.)

The TAC case for licensing was pretty straightforward.  Had permanent licenses in about a day after providing the contract number that showed SWSS.

I held of installing the permanent licenses until after hours in the event that something would go wrong and take the system down (still nervous after the CCX incident).  Installation went fine with one side issue.

I had complaints about SpeechConnect / voice enabled directory handlers on Unity Connection not working right.  Turns out CUC didn’t like the permanent licenses as far as SpeechConnect.  It had pulled the licenses from ELM/PLM properly and was in compliance, but it took a restart of the Conversation service for it to start doing the voice recognition stuff again.  Rather odd.

Holding at CCX 10.6

Since 10.6 is the last version of CCX to support CAD/CSD and Finesse, I’m working to migrate the contact center over to Finesse.  There are some usability complaints we’re working through.  The users love the idea of a dedicated app that pops when a call comes in as well as the agent-to-agent chat inside CAD.  Getting them to use a web-browser for Finesse has been a challenge.   Once I have those details ironed out we’ll force them into Finesse when we upgrade to CSR 11.5 in the summer.

 

 

 

Deploying Multistream Conferencing with vTS and CUCM

With the release of CE8 code for Cisco video endpoints (like the SX10 (8.1), SX20 (also MX200-G2 and MX300-G2) and SX80-based endpoints like the SX80, MX700 and MX800), and the appropriate infrastructure components, multistream video is a possibility.  Multistream video allows an endpoint to send multiple resolution video streams and have the bridge pass the most appropriate streams to the far-end video units.  The far end video unit would receive a full resolution stream of the active speakers, and then low quality streams of the other participants.  The most useful feature of multistream video is the ability to use both screens of a dual-screen video unit to see remote participants (when doing single-stream transcoded mode, you can only do single screen video, and secondary screen content.)  Multistream also allows for ActiveControl layout, which allows the endpoint to choose the video layout vs. the video bridge determining the layout of the participants (which has rudimentary DTMF layout control).

Components used in my lab configuration:

• SX10 (CE8.1)
• MX800 (CE8.0.1)
• (2) 8845 video phones (used to inject more video streams — these endpoints do not support multistream, they do single stream and receive their layout from the bridge)
• Conductor XC4.1
• vTS 4.2(4.23)
• CUCM 11.0(1)21900-11 (Latest and greatest version is a requirement) -or- VCS X8.7.1

This guide assumes you’ve already setup a Rendezvous (aka MeetMe) number/URI that is routed to Conductor/vTS and you’re able to to normal conference calls.  We’ll modify settings to enable multistream.

Guide to configure endpoints and CUCM SIP Profile – http://www.cisco.com/c/dam/en/us/td/docs/telepresence/infrastructure/solutions/cmrpremises/cmr-premises-deployment-guide-r6-0.pdf

The relevant portion of this configuration is to make sure your SIP trunk to conductor is in a Location that supports full quality video.  I sent the inter-region bandwidth to UNLIMITED in my test system.  Cisco recommends a minimum of 1mpbs per screen, otherwise the vTS bridge may kick that video unit down to single-stream transcoded mode.

Configure the endpoint to support multistream

In CUCM the setting is in the device specific settings, Multistream Mode needs to be set to Auto.  Despite some of the documentation reading otherwise, Auto will attempt to do multistream, there is not actually an On setting.

Configure CUCM

Configure the SIP Profile used by the SIP trunk to Conductor to include the following settings:

• Allow iX Application Media and Allow multiple codecs in answer SDP are checked on.
• SDP Transparency Profile is set to Pass all unknown SDP attributes

In System > Service Parameters > Call Manager Service > click advanced > set SIP Maximum Incoming Message Size to 18000.

Configure Conductor

On the Conductor server, under the Conference Template you’re using for your conference, select advanced template parameters and add:

• Enable iX protocol – True and the box checked
• Multiscreen layout – ActivePresence and the box checked

No settings on vTS need to be changed, it will automatically do multistream if the endpoints meet the requirements, and CUCM (or VCS) and Conductor are properly configured.

When you join with a multi-stream endpoint you will see the following on vTS Conferences page:

 

You’ll notice the endpoints that support multistream show Multistream, and the 8845 phone named “Mike White” is Standard because it only supports a single stream.

If we look at the statistics for 5580 (the SX80) you’ll see multiple video streams being sent and received:

 

Lastly if we look at the call statistics from the video endpoint itself, we see the same information:

 

The touchpanel now shows more details in the layout.  You can see each participant in the conference and the active speaker.

 

While you can select from several canned layout modes (same typical layouts are you’re used to), this version doesn’t yet support complete drag and drop layout of individual participants where you want them.  If you select a particular participant, you can see information about any of the participants and boot them if you are meeting organizer:

 

Overall its very cool, and sets the groundwork for much more flexibility in the future with layout control.

 

Registering an SX-10 (and TC-based endpoints) through Collab Edge MRA

Recently I worked on an MRA deployment using SX-10, MX-300 and DX-series (650/70/80) endpoints.

I had Expressway-C and E working successfully for 8800 series MRA, but needed to get the TC-based and DX-based endpoints to register.  This turned out to to involve some issues that I wasn’t expecting.

TC-based endpoint registration

There isn’t a lot of documentation  for TC endpoint registration through MRA (since traditionally it’s been registered to VCS through VCS-E).  The best documentation that I could find was here:  http://www.cisco.com/c/en/us/support/docs/unified-communications/expressway/118696-config-cucm-00.html

I began the deployment by registering the MX-300 I use directly to CUCM to make sure I had it successfully working before attempting registration through MRA.  I’d previously had it registered to a VCS-C.

MRA requires TC 7.3 code, so I decided to deploy 7.3.4 since it is the latest bugfix version.  I downloaded and deployed the COP file to CUCM since CUCM will be in control of what version of software the TC endpoint will use once it’s registered to CUCM (like typical phones get firmware).

Note that TC 7.3.3 or greater firmware have different functionality for remote screen monitoring of systems!  TC 7.3.3 introduces the requirement to have an option key for remote system/screen monitoring of the TC endpoint.  You’ll need to work with TAC/Cisco to get option keys cut if you are doing remote screen monitoring before going to 7.3.3+.

Registration to CUCM was straightforward.  I defined the device on CUCM as you would normally define a phone, picking the appropriate SIP profiles.  I didn’t do secure registration as this is optional.   (The documentation above does mention that secure registration is optional, but the example works through a secure registration.)

I made sure to set the device association my end user in CUCM.  This is important for MRA later.

Once the MX-300 was registered and making calls successfully through CUCM, I moved it out to a general internet connection to work on MRA.

On the touchpanel I launched the Provisioning wizard and selected Cisco UCM via Expressway.  After putting in my credentials I was greeted with this error:

After doing quite a bit of research and looking at the detailed error logs from the MX-300, it turns out that your Expressway-E certificate must also include a SAN for the domain name itself (e.g.  yourdomain.com).  The error actually indicates that it wants a collab-edge.yourdomain.com SAN:

Edge TLS verification failed: Edge domain ‘yourdomain.com’ and corresponding SRVName ‘collab-edge._tls.yourdomain.com’ not found in certificate SAN list.

The challenge I had is that the certificate I’d bought from GoDaddy for Expressway-E (that was working with 8800 MRA) wasn’t a UCC or multi-SAN certificate and you need to have at least the Expressway-E as the CN and the domain as a SAN.

At this point I decided that it was time to move from GoDaddy to DigiCert since they have unlimited resigning of certificates without having to revoke any of them.  This essentially allows you to create as many certificates as you want without having to keep buying more like GoDaddy.  I bought the Wildcard Plus certificate and used it to create a multi-SAN certificate for my Expressway-E.  The CN is always *.yourdomain.com, but you can add a bunch of SANs (like 20 or so.)

I generated the certificate with the following SANs – edge.yourdomain.com, yourdomain.com, collab-edge._tls.yourdomain.com and _collab-edge._tls.yourdomain.com.   One of the partner engineers that I talked to said that he got it working without having to add either collab-edge SAN.  (I’ve not looked into why/when we’ll need the collab-edge SAN and if it is actually be collab-edge as the error indicates, or if it needs the preceding underscore on collab-edge like the SRV records have.)

After applying the certificate to Expressway-E and rebooting it I tried the provisioning wizard on the touchpanel again and was greeted with the SAME ERROR.

It turns out that I hadn’t included the DigiCert root and the DigiCert Intermediate cert in the list of Trusted CAs on the endpoint itself.  The documentation indicates how to install it here – http://www.cisco.com/c/en/us/support/docs/unified-communications/expressway/118696-config-cucm-00.html#anc10  Make sure you have both root and intermediate (if the CA you used signs with an intermediate) on the TC endpoint.

After this the MX-300 registered like a champ and is able to do calls.  I followed this same process to get an SX-10 registered as well.

The partner engineer I talked to said he had to work through a couple issues on his test:  1) Endpoint rejecting the user credentials when running through the provisioning wizard.  Make sure the endpoint is associated with the End User and that the end user has CTI Enabled and CCM End user.   2) Getting an http download error after getting through the initial expressway authentication and that it was caused by the endpoint needing to do a _cisco-uds.yourdomain.com lookup internally to find out where CUCM is to download it’s configuration.   He was in a split domain situation and didn’t have a _cisco-uds record for the external domain on the inside.

I’ll detail the adventure for the DX-series on another post.

 

 

 

 

Expressway 8.5.2+ MRA Issues

I’ve run into a couple challenges after an upgrade to Expressway/VCS 8.5.2 where MRA for phones quit working.

I found a bug that broke MRA in 8.5.2 (the recommendation has been to downgrade back to 8.5.1).  That bug is shows that it is now fixed in 8.6, so I upgraded to 8.6.1 recently.  MRA started working again, but only on one out of every three login attempts.  It was really weird.  In looking at the logs it showed a bunch of errors:

Home CUCM not available – Unknown CUCM cluster for node sub02

Home CUCM not available – Unknown CUCM cluster for node sub03

The deployment I was working on is a three node (pub and two subs), running split DNS (different internal domain than the external domain name).

After a lot of digging it turns out that there is a change in the way MRA handles CUCM lookup.  When I installed the system added my pub and subs to Expressway-C by IP address.  But it looks like Expressway now attempts to communicate with them via hostname, and not IP as they were defined by me.

Since Expressway is using the domain suffix assigned to MRA (extdomain.com), it is attempting to lookup sub02.extdomain.com and sub03.extdomain.com.  I didn’t have A records for these on my internal DNS server extdomain zone since I’ve never needed to resolve them by the external domain name.

Adding these two records fixed the login issue and it now logins on first attempt like it used to.

Collab Edge MRA for 7800/8800/DX Series Endpoints

Cisco recently posted Expressway (and VCS) X8.5.2 and 10.3.1 firmware for the 8800 and 7800 series phones.  The combination of these products allows these phones to register remotely to CUCM utilizing Collaboration Edge MRA.  (The DX-series (650/70/80) is expected to support MRA in the next release of code due out shortly.)

This functionality isn’t TAC supported yet, and has been released in a “feature preview” form.  I’ve set it up and tested it and it works well for the most part.  However, there is not full feature parity for a phone registered via MRA vs directly registered to CUCM, but for testing and basic calls, it works well.

In order to set it up, make sure your Expressway MRA deployment for Jabber is working properly.  MRA for the 7800/800 series phones uses the same service discovery process that Jabber uses, so if you have Jabber working, you’ll have 95% of the work done.

One important piece of information to know is that the phone firmware trusts 100+ public root CA certificates.  If your Expressway-E server does not have a certificate signed by one of these CA’s it’s not going to work for the phones.

Here’s the basic procedure I followed to make it work:

1. Installed the 8800 10.3.1 COP file via OSadmin and restarted the TFTP service.
2. Logged in to Jabber via MRA to ensure the correct functionality of my MRA system and my login credentials.
3. Defined the phone in CUCM and then connected the phone directly to CUCM so that it would pull the version of firmware that supports MRA.  (My 8851 phone shipped with an older version of code that did not have MRA support.)
4. Took the phone off of the corporate network to an internet-access only network.
5. I had initial problems with the phone not attempting MRA lookup after being connected to the internet-only network, so I followed the troubleshooting process of resetting the Network settings on the phone.  It then started to try the MRA process.

Steps the phone follows in MRA registration:

1) Phone attempts normal TFTP registration/_cisco-uds._tcp.domain.com lookup process:

This fails because the phone has no direct access to CUCM.

2) Firmware now prompts for MRA credentials (These would be the same credentials you use for Jabber MRA login — in my case it is set to use LDAP/AD for authentication):

Phone now attemps  _collab-edge._tls.domain.com service record lookup (like Jabber does) to discover the Expressway-E/VCS-E host.

3) Phone completes MRA login process

The phone is now registered and usable.

I’ve read conflicting information about the number of calls supported, and number of lines supported via MRA.  In my experience I have two lines on the phone registered and am able to make two calls per line.  (I’ve not tested more than two calls per line.)  The list of features that may work or not is extensive, so be careful as things like Barge or Intercom may not work yet.

The phone also upgraded code via MRA successfully which is good to know.

I’ve noticed some oddities with on-hook vs. off-hook dialing.  I know there are some limitations around KPML currently.  In my experience it seems to off-hook dial fine on the primary line, but on a secondary line or when attempting a second call on the primary line you MUST on-hook dial.

Phone registration isn’t supported via TAC yet so feel free to post here and we can collectively attempt to assist.  Remember the most basic step to troubleshoot is to see if your Jabber can successfully login.

Collaboration Edge (MRA) with Split DNS Domains

I recently helped one of our customers work through a split DNS domain Collab Edge MRA deployment that was quite educational.  Here are a few tidbits that I picked up.

Scenario:

Customer’s external (internet facing) DNS domain is:  collabdomain.com
Customer’s internal domain is:  internaldomain.local  (not resolvable externally, only served by the internal DNS server)

Expressway-E lives on the internet so it’s hostname would be vcse.collabdomain.com

Expressway-C, CUCM, IM&P (CUP) and Unity Connection live on the internal domain, so they would be called ucm1.internaldomain.local, cup1.internaldomain.local, ucxn.internaldomain.local.

The users accounts would live in internaldomain.local (presumably the LDAP user would be user@internaldomain.local, etc.)

The external DNS server would serve the public collabdomain.com zone.

The internal DNS server would serve internaldomain.local, as well as an internal version of the collabdomain.com zone.

The challenge with these separate domain names is that when a user goes to login to Jabber via MRA, the service discovery process is going to fail because they are going to type in their IM address as user@internaldomain.local, and there is no _collab-edge._tls.internaldomain.local SRV record availble on the itnernet (nor could there be).

To get around this, the jabber-config.xml file can be modified to include the external domain name that Jabber should try to do service discovery against. 

<Policies>
  <VoiceServicesDomain>collabdomain.com</VoiceServicesDomain>
</Policies>

Jabber must first login internally (directly on your corporate network) to pull down the jabber-config.xml that includes this policy.

With this configuration now loaded in Jabber, when the user attempts to login from outside via MRA they will still use their user@internaldomain.local credentials, but Jabber will use the specified VoiceServicesDomain — in this case — collabdomain.com.  It will do the SRV record lookup on _collab-edge._tls.collabdomain.com and now be able to resolve Expressway-E and login.

In addition to the the normal _cisco-uds._tcp.internaldomain.local SRV record, Jabber needs another _cisco-uds._tcp.collabdomain.com SRV record (on the internal DNS server) pointing to ucm1.internaldomain.local for MRA to work with the split domains.The one criticism to this method is requiring the Jabber client to login internally first, which isn’t an ideal scenario for large distributed organizations.  One method that is supposed to work around this is to prime Jabber by installing it generically and launching it from an URL:ciscojabber://provision?ServicesDomain=internaldomain.local&VoiceServicesDomain=collabdomain.com

This could be provided in an email and will supposedly launch Jabber so that it tries the external domain without having to ever login internally first.