Introduction

 

After a few days playing with the IOx platform on a Connected Grid Router (CGR), I decided to write an informal post with my experience and lessons learned during this initiative. The main objective here is to describe the overview of the platform and also provide no-brainier quick start configuration to have everything up and running.

 

Before we start, be aware that IOx is evolving very fast and this post could be outdated at the time you are reading. At the time I'm writing, the latest software version public available for CGR1240 is 15.4(3)M2 (cgr1000-universalk9-bundle.SPA.154-3.M2.bin). Everything described here is taking this scenario as baseline.


Fog Computing and IOx Platform

 

The huge increasing amount of data generated due growth of Internet of Everything (IoE) is becoming a challenge for the applications and infrastructure, that need to scale accordingly in order to be able to process them. To be successful we need to take computational resources to the edge of the network, near where the end-users/devices are connected, providing a way to compute and aggregate information before send it back to the Cloud. This concept is called Fog Computing, and it is the motivation behind the Cisco IOx platform.

 

CGRAppHostingOverview.jpg

 

The IOx platform disaggregates the IOS network operating system from the device's hardware through a specialized hypervisor technology. This hypervisor is capable of hosting the IOS and also a Linux virtual machine called Guest OS (GOS), which  shares hardware resources. The communication between GOS and IOS, or external network, uses GigabitEthernet 0/1 interface of IOS as gateway. The Linux running on Guest-OS is based on Yocto Project, an embedded Linux distribution.

 

The amount of resources available for the GOS depend on which platform you are running IOx. On CGR1240 device, the GOS virtual machine specification have 192MB of RAM and 1GB of HDD, sharing the Intel(R) Atom(TM) CPU E640 processor. In order to determine which platforms are currently supporting IOx, and what are the GOS capabilities on them, please check the platform documentation.

 

Before You Start

 

If your CGR is running CG-OS software, such as "cgr1000-uk9-final.5.2.1.CG4.3", you will need to migrate it to IOS (IOx-based) software. This process is hard-working but not that difficult. Our recommendation is to contact the Cisco Technical Assistance Center (TAC) and ask for support of a specialized engineer. However, if you feel brave and understand the risks check this:


Cisco Communities - Migrate CGR 1240 from CG-OS to IOS

 

If your CGR is already running IOS (IOX-based), such as "cgr1000-universalk9-bundle.SPA.154-3.M1.bin", just make sure to upgrade your IOS to the newest version in order to get new features and bug fixes. To accomplish that, just follow:

 

     1. Download proper software version from Cisco.com and put it on a FTP/TFTP server reachable from your CGR:

 

     Release 15.4.3M2 - http://cs.co/9006FJij

 

     2. CGR's flash partition:

 

CGR1240-IOX# copy tftp://192.168.1.100/cgr1000-universalk9-bundle.SPA.154-3.M2.bin flash:























 

     3. After copy it, install using the bundle command. This will install all 3 components of IOx (hypervisor, IOS, Guest OS):

 

CGR1240-IOX# bundle install flash:cgr1000-universalk9-bundle.SPA.154-3.M2.bin























 

     4. Save the config and reload the device.


CGR1240-IOX# write
CGR1240-IOX# reload





















 

     PS: As said in step 3, the binary image contains all the 3 components images bundled together. In case you need access to individual components (hypervisor, IOS or GOS) images, you can use GNU Tar command to unpack it from Linux or Mac OS. These files are handy in case you need to recover from ROMmon or change component images individually.


MACBOOK:~ davigar$ tar xvf cgr1000-universalk9-bundle.SPA.154-3.M2.bin
















Validate IOx Installation


In order to validate the proper IOx environment, we can issue two commands to confirm all components are installed properly. The first validates the IOx hypervisor version currently installed:

 

CGR1240-IOX# show platform hypervisor
version: 1.1.0














The second validates the IOx GOS image version currently installed and the status. Don't worry to see running even though you didn't configure anything yet, but IOx starts GOS as soon as it boots. The second command is:

 

CGR1240-IOX# show platform guest-os
Guest OS status:
Installation: Cisco-GOS,version-1.26
State: RUNNING














Configure IOS to provide Networking for GOS


There are multiple ways to configure IOx in terms of networking between IOS and GOS, however I will describe one based on IPv4 with DHCP and NAT. On this scenario, the GOS will receive an IPv4 network configuration, including address, DNS and gateway, through DHCP server configured on IOS.

 

Lets start configuring interface GigabitEthernet0/1 with a private IP address from 192.168.1.0/24 segment, which will be the default gateway for the GOS. Even though we are not using IPv6, we need to enable it on the interface because this is used to management services. Also, we will configure it as a NAT Inside (private) interface. After get in configuration mode, the configuration should be done like:

 

interface GigabitEthernet0/1
  description Internal Gateway for Guest-OS
  ip address 192.168.1.1 255.255.255.0
  ip nat inside
  ip virtual-reassembly in
  duplex auto
  speed auto
  ipv6 enable
  no shutdown
















The interface which is used by CGR as exit to the external network (WAN, for example) need to have the NAT Outside (public) configured in order to have NAT working properly. In my case I'm using GigabitEthernet2/1 as WAN, so the configuration should be:


interface GigabitEthernet2/1
  description WAN Interface (Lab MGMT)
  no switchport
  ip address 172.16.1.201 255.255.255.0
  ip nat outside
  ip virtual-reassembly in
  duplex auto
  speed auto
  no shutdown















 

The DHCP configuration is simple. In my case the GigabitEthernet0/1 has IP 192.168.1.1 from 192.168.1.0/24 segment, which will be the default gateway of GOS. Also, my DNS servers are 172.16.1.50 172.16.1.51. I configured DHCP to remember the bindings and lease time for 5 days, but this is not required. The configuration should looks like:


ip dhcp pool guest-os
  network 192.168.1.0 255.255.255.0
  default-router 192.168.1.1
  domain-name coibr.cisco.com
  dns-server 172.16.1.50 172.16.1.51
  lease 5
  remember
















The last piece of configuration is the NAT. In my case I got an IP from my GigabitEthernet2/0 interface's segment, 172.16.1.0/24, and created a static NAT between GOS private IP address (192.168.1.2) and an public IP address (172.16.1.202). The configuration is:

 

ip nat inside source static 192.168.1.2 172.16.1.202














 

After that you just need to exit configuration mode and restart GOS. The boot can take around 5 minutes. The command is:

 

CGR1240-IOX# guest-os 1 restart
Restarting Guest OS ......... Done!














 

Initial Configuration of GOS

 

After configure the IOx networking, we need to validate if the environment is UP and RUNNING before configure it. The most important command that I used was:


CGR1240-IOX# show iox host list detail

IOX Server is running. Process ID: 301
Count of hosts registered: 1

Host registered:
===============
    IOX Server Address: FE80::D68C:B5FF:FEA2:F049; Port: 22222

    Link Local Address of Host: FE80::D68C:B5FF:FEA2:F04C
    IPV4 Address of Host:       192.168.1.2
    IPV6 Address of Host:       fe80::d68c:b5ff:fea2:f04c
    Client Version:             0.1
    Session ID:                 1
    OS Nodename:                CGR1240-IOX-GOS-1
    Host Hardware Vendor:       Cisco Systems, Inc.
    Host Hardware Version:      1.0
    Host Card Type:             not implemented
    Host OS Version:            1.26
    OS status:                  RUNNING

    Interface Hardware Vendor:  None
    Interface Hardware Version: None
    Interface Card Type:        None

    Applications Registered:
    =======================
    Count of applications registered by this host: 0









On the output above you can see many information about the GOS, including its link-local IPv6 internal management address and also its IPv4 private address, received from IOS. The hostname is dynamically assigned, adding the prefix "-GOS-1" to the CGR hostname.


We are able to execute commands from IOS into the GOS image, and this is required for initial GOS setup. The first configuration is redefining the administrative account (root) password. In the command below I'm defining root's password as "cisco123":


CGR1240-IOX#iox host exec "resetpw cisco123" CGR1240-IOX-GOS-1
  Sending command to IOX Client and waiting up to 300 seconds for response...
Password reset successfully.








The second and last configuration is enabling the SSH service inside GOS. For that, we need to issue the command below:


CGR1240-IOX#iox host exec enablessh CGR1240-IOX-GOS-1
  Sending command to IOX Client and waiting up to 300 seconds for response...
ssh enabled successfully.








After that, you should be able to log in by SSH, using private IP address of GOS from IOS, or using NATed public address from outside networks connected to CGR.


GOS Connectivity Troubleshooting

 

If you get in trouble during the processes described previously, there are a few commands that may help you understand what is going on. The first one is to determine which IP address the GOS received from DHCP, which should be the first one:

 

CGR1240-IOX#show ip dhcp binding
Bindings from all pools not associated with VRF:
IP address          Client-ID/             Lease expiration        Type
                    Hardware address/
                    User name
192.168.1.2         01d4.8cb5.a2f0.4c       Mar 08 2015 05:34 PM    Automatic






 

The second validation is to ping and arp between IOS to GOS private address. This will validate internal communication, which should be simple back-to-back scheme. The commands:

 

CGR1240-IOX#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/8 ms

CGR1240-IOX#show ip arp 192.168.1.2
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  192.168.1.2             0   d48c.b5a2.f04c  ARPA   GigabitEthernet0/1






 

The third validation is to verify if NAT is working properly. For that, test a few connections (ICMP, SSH) from networks outside CGR and verify if the translations are done. The command:

 

CGR1240-IOX#show ip nat translations
Pro Inside global         Inside local          Outside local         Outside global
tcp 172.16.1.202:22       192.168.1.2:22        10.134.15.122:56242   10.134.15.122:56242
udp 172.16.1.202:123      192.168.1.2:123       69.41.163.31:123      69.41.163.31:123
udp 172.16.1.202:51294    192.168.1.2:51294     10.97.20.54:53        10.97.20.54:53
icmp 172.16.1.202:53590   192.168.1.2:53590     10.134.15.122:53590   10.134.15.122:53590
--- 172.16.1.202          192.168.1.2           ---                   ---






 

Conclusion

 

Now it's time to play and having fun! Please understand that this is just a very simple use case, where you uses GOS as a Linux virtual machine inside CGR. However you could also develop an application and pack it into a bundle and install on GOS through IOS, but this is a theme for another talk... ;-)

 

root@CGR1240-IOX-GOS-1:~# uname -a
Linux CGR1240-IOX-GOS-1 3.8.11-yocto-standard #1 SMP PREEMPT Thu Jun 5 22:46:43 PDT 2014 i686

root@CGR1240-IOX-GOS-1:~# ip neighbor
fe80::d68c:b5ff:fea2:f049 dev eth0 lladdr d4:8c:b5:a2:f0:49 REACHABLE
192.168.1.1 dev eth0 lladdr d4:8c:b5:a2:f0:49 REACHABLE

root@CGR1240-IOX-GOS-1:~# ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether d4:8c:b5:a2:f0:4c brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 scope global eth0
    inet6 fe80::d68c:b5ff:fea2:f04c/64 scope link
       valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop state DOWN
    link/sit 0.0.0.0 brd 0.0.0.0

root@CGR1240-IOX-GOS-1:~# ip route      
default via 192.168.1.1 dev eth0
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.2

root@CGR1240-IOX-GOS-1:~# python
Python 2.7.3 (default, Jun  5 2014, 22:58:09)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print("I'm alive inside CGR!")
I'm alive inside CGR!






 

References