ST 2110 NICs

Norsk receives and sends ST 2110 with MTL, which drives a NIC directly from user space. That needs a few things from the host that an ordinary container never gets. norsk-ctl works out which NICs can do it, lets an instance claim them, and gives that instance’s media container exactly what MTL needs, and nothing more.

Which NICs count

norsk-ctl uses the Norsk engine’s own rule. A NIC is ST 2110-capable when it has:

• a DPDK-capable driver;
• a PTP hardware clock;
• a link of 10 GbE or more.

Today only Mellanox / NVIDIA ConnectX (mlx5) NICs can be launched. Other capable NICs (Intel ice, …) need VFIO and are listed, but cannot be claimed yet.

Host setup

For each ST 2110 NIC:

1. RDMA verbs. Install rdma-core, so the NIC has a device under /dev/infiniband/uverbsN (the mlx5_ib module).

2. A hugetlbfs mount at /dev/hugepages-st2110-<ifname>. Each NIC’s MTL takes this mount to itself. For example, for mellanox0:

   sudo mkdir -p /dev/hugepages-st2110-mellanox0
   sudo mount -t hugetlbfs -o pagesize=1G nodev /dev/hugepages-st2110-mellanox0
   # and in /etc/fstab:
   # nodev /dev/hugepages-st2110-mellanox0 hugetlbfs pagesize=1G 0 0

3. Hugepages reserved on the NIC’s NUMA node, enough for MTL. If the instances also share a Norsk Domain with a hugepage pool, reserve for both, and bind the domain’s pool to the same node.

Runtime → Infrastructure shows a card per ST 2110 NIC: its PCI address, NUMA node, RDMA device and mount. A NIC that is not set up lists the step still missing. GET /api/hardware reports the same, as st2110Nics.

Note

PTP is the network’s job. The NIC needs a PTP hardware clock to count as capable, but norsk-ctl does not check that the host is locked to a grandmaster.

Claiming NICs

An instance claims the NICs it runs ST 2110 on:

norsk-ctl instance launch-template capture --template chanel-capture --st2110-nics mellanox0

• In the UI, the Launch form lists the host’s NICs when the template declares any. It pre-selects the template’s own, and disables any that are not set up or that another instance holds, saying why.

• The launch is refused, before anything starts, when a claimed NIC:

  • is not on this host;
  • is not set up (ST2110_NIC_NOT_LAUNCHABLE, with the fix);
  • is held by another instance (ST2110_NIC_IN_USE).

  A NIC runs one MTL at a time, so a claim is exclusive until the instance is deleted.

• A claim puts media on the host network (network mode hybrid), because only the host’s network namespace has the NIC. Claiming NICs with network mode docker is refused (ST2110_REQUIRES_HOST_NETWORK).

Claiming NICs is a host-privileged launch setting, like choosing hardware.

What a product template can declare

• advanced.st2110Nics.default names the NICs a template’s instances claim by default. This suits a customer turnkey that knows its host. A launch can override it, or decline it with an empty list.
• requirements.st2110Nics: { count: N } says the product needs N NICs but cannot know which. The operator picks them at launch, and a launch claiming fewer is refused (ST2110_NICS_REQUIRED), listing the NICs that are free.

What the media container gets

For an instance that claims NICs, and only then:

Network	the host network
Capabilities	NET_ADMIN, SYS_RAWIO, IPC_LOCK, SYS_NICE, on top of Docker’s defaults
Devices	each claimed NIC’s /dev/infiniband/uverbsN
Volumes	each claimed NIC’s /dev/hugepages-st2110-<ifname>
Environment	NORSK_ST2110_NICS: a JSON list of { ifname, pciBdf, numaNode, hugeDir, ipAddresses, cores }

The product builds its NIC configuration from NORSK_ST2110_NICS, rather than hard-coding host facts into its workflow. The Studio container gets the same variable, for a component that binds the NICs through the SDK, but none of the privileges.

cores is the NIC’s own share of its NUMA node’s CPUs, isolated cores first, for MTL to pin its scheduler to. The 2110 NICs on a node split its CPUs between them, so instances on different NICs never pin the same cores; Norsk’s recommendCoresFor picks by node alone, and two instances using it on one host starve each other. A product takes as many as it needs from the front of the list. It is empty when the NIC’s node is unknown, and absent from a ctl that predates it.

Nothing else is needed:

• no --privileged;
• no seccomp change;
• no memlock limit.

The capability list is fixed by Norsk’s ST 2110 extension, which refuses to start without all four.