Testing out networking infrastructure and building working PoCs for a new environment can be tricky at best and downright dreadful at worst. You may run into licensing requirements you don’t meet, or pay pricey fees for advanced hypervisor software. Proprietary network systems can cost hundreds or thousands of dollars just to set up a test environment to play with. You may even be stuck testing on physical, perhaps outdated hardware you must hook up yourself, and if you need to test with more hardware, you must find and buy more.

Why SONiC?

Enter SONiC (Software for Open Networking in the Cloud), the free, community-developed, Linux-based network operating system (NOS) hardened in the data centers of some of the largest cloud service providers. SONiC is an ideal choice for centers looking for a low-cost, scalable, and fully controllable NOS without sacrificing flexibility or security. It offers all the standard networking functionality developers should need and is constantly evolving with new features and updates.

Community SONiC is open-source, eliminating licensing and vendor lock-in walls and significantly reducing costs compared to a proprietary NOS. This also enables intense customization. You can build a custom image of SONiC with all the parts you need and none that you don’t:

Reduce unnecessary complexity and resource usage
Aid in automation
Deploy the same image to hundreds of switches, all pre-configured and ready to work
Enable ZTP
Install certain Docker containers
Configure routing
Build and add your own proprietary features

For more information, see Building an NVIDIA Pure SONiC Image.

NVIDIA Air and SONiC

The NVIDIA Air platform offers a true infrastructure-as-code (IaC) solution. NVIDIA Air makes it so easy to instantly generate virtual data center replicas with hundreds of switches and servers. With NVIDIA Air, you can test as you please. When you irrevocably destroy something, spin up another copy.

Best of all, you don’t have to break the bank. NVIDIA Air is free for anyone and everyone.

One of the amazing features of NVIDIA Air is the ability to spin up premade, fully configured labs on a dime. NVIDIA has created a new, preconfigured SONiC lab, equipped with the latest GA release of community SONiC (202305 at the time of publication) in the classic spine-and-leaf architecture found in modern data centers. Anyone can try out the magic of SONiC on NVIDIA Air effortlessly.

SONiC lab

The SONiC BGP EVPN VXLAN lab demo on NVIDIA Air is a new lab consisting of a standard spine-and-leaf topology running SONiC switches with Ubuntu servers connected to leaves to demonstrate VLAN connectivity.

The lab walks you through its configuration and how it was automatically provisioned with Ansible. It also shows you how to interact with it. You can find all the documentation, configuration files, and Ansible scripts on the public /cumulus-consulting GitLab repo.

It demonstrates the following fabric:

BGP underlay fabric using BGP numbered interfaces
VXLAN & EVPN overlay encapsulation data plane and overlay control plane, respectively

You can also use the lab as a base and further configure it to your needs, using NVIDIA Air features such as the following:

Full CLI and API functionality
Control plane software including BGP, VLANs, and containers
Automation and Zero Touch Provisioning (ZTP)
Network monitoring with streaming telemetry
Interop testing between NVIDIA Cumulus Linux and SONiC
Custom topologies and network designs

SONiC switch configuration

The lab uses the SONiC image specifically built for virtual devices, called sonic-vs. This image is already installed on all the SONiC switches in the lab. No need to boot into ONIE or install the image yourself.

SONiC architecture is containerized. Programs with similar or dependent functionalities are placed in modules together, and each module is placed in an independent Docker container.

Central configuration is managed by a redisDB instance referred to as ConfigDB. Device metadata, interfaces, ports, VLAN configuration, and much more are handled in ConfigDB. ConfigDB can be manipulated in multiple ways, including editing the config_db.json file manually or using the SONiC CLI. For more information, see the SONiC Configuration Database Manual.

Typically, the SONiC CLI is used for building a test environment and proof of concept (PoC), while managing config_db.json is performed in production environments.

Split-unified mode

split-unified mode is implemented for this lab. This mode enables separate switch and routing configurations.

Switch configuration is handled by ConfigDB, while routing is configured with FRRouting (FRR) and placed into a single etc/sonic/frr/frr.conf file. The option is enabled by inserting the following on the SONiC switch:

"DEVICE_METADATA": {
        "localhost": {
            ...
            "docker_routing_config_mode": "split-unified"
        }
    }

The source code for the lab offers the same configuration for split mode, where the switch and routing are still configured separately as it is in split-unified. However, in split mode, each FRR daemon is contained in its own config file, instead of one single file.

Depending on your environment, you may find this easier to manage, but typically split-unified is implemented as it only deals with one file. For more information, see the configuration files on GitLab.

Routing configuration

Routing configuration is handled by FRR, separate from the switch configuration. FRR is a fully featured, high-performance, free software IP routing suite for Linux and Unix platforms. It implements all standard routing protocols such as BGP, RIP, OSPF, IS-IS, and more, as well as many of their extensions. The lab uses FRR to implement numbered BGP EVPN as the control plane.

By default, FRR is pre-installed on SONiC and is available when the switch boots.

There are two ways to configure routing using FRR:

Manually edit configuration files
Get familiar with VTYSH

VTYSH enables you to use all the CLI commands provided by FRR in a single, integrated shell. The lab has been configured using VTYSH.

To enter into VTYSH:

admin@leaf01:~$ vtysh

Hello, this is FRRouting (version 8.5.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

To make configurations, enter into config mode:

leaf01# configure
leaf01(config)#

Then, enter your configurations:

leaf01(config)# router bgp 65101
leaf01(config-router)# bgp router-id 10.0.0.1
leaf01(config-router)# neighbor 172.16.1.0 remote-as 65199
leaf01(config-router)# address-family ipv4 unicast
leaf01(config-router-af)# neighbor 172.16.1.0 activate
leaf01(config-router-af)# network 10.0.0.1/32
leaf01(config-router-af)# end

To save your configurations, use write:

leaf01# write
Note: this version of vtysh never writes vtysh.conf

Warning: attempting direct configuration write without watchfrr.
File permissions and ownership may be incorrect, or write may fail.

Building Configuration...
Integrated configuration saved to /etc/frr/frr.conf
[OK]

Get started

SONiC is backed by a rich, active community and regularly contributed to by the networking industry experts at NVIDIA, who are fully committed to further evolving the open-networking experience that SONiC offers for all.

In the future, along with general updates and new features, NVIDIA plans to implement a bring-your-own-SONiC image infrastructure to NVIDIA Air, so that you can take full advantage of the customizability SONiC offers within the free, convenient environment that NVIDIA Air provides.

Ready to give SONiC a try? Head to NVIDIA Air and spin up a copy of your own today.