@rktidwell @rdimitrov @nickolaev @edwarnicke @davecremins Can you please take a look? It's a rough draft as we discussed on the weekly meeting. Mostly describes current implementation and assumptions, and more importantly lists the challenges/open issues.

If I get it right, I am a big NO here! I am sorry but I don't see the NSM spirit and concepts applied here, instead, this spec os proposing to tweak and patch a ton of code just to be able to request a simple interface.
And all this is because we are supposedly getting scheduling based on how DP handles resources. That is fine, everything else is not. Let me point:

• Kubelete will insert resources ONLY at pod creation time. This is not what NSM is about. Use Multus instead
• We are planning to have random env variables read in the init containers etc. That is wrong - no forwarder specific code should exist beyond its binary.
• annotating containers so that DP can pick nodes for us

The workflow should rather be:

• I run my pod (Client or Endpoint). I want to request a host resource (interface or device) so I declare my mechanism preference.
• The forwarder is selected and picks my request, finds my resource or allocates it from the PF if that is the case (i.e. the device support SR_IOV)
• The forwarder injects that resource in the pod (interface or device) no matter if this is pod creation or during runtime

I think we should drop the whole Device Plugin idea and figure out a solution independent from it. Otherwise we will have to bend our principles and ideas and we endup with a Multus variation which is not the point at all with NSM.

Just a small comment so we're all aware - scheduling isn't the main reason for using device plugin, main reason is that it already provides a ton of features like device discovery with selectors for vendors/drivers/device types, health checking of the links, grouping devices into the resource pools, finally allocation the device resources themselves based on what needs to be allocated, I think you may be missing a good number of benefits. And dropping DP results in a need of re-implementing almost all of this in NSM.

I assume you are referring to this plugin:
https://github.com/intel/sriov-network-device-plugin

If that is the case, can we just use whatever lives here:
https://github.com/intel/sriov-network-device-plugin/tree/master/pkg

And replace this part with an NSM forwarder logic:
https://github.com/intel/sriov-network-device-plugin/tree/master/cmd/sriovdp

I mean, the SRIOV plugin has already what is needed, that is not the argument here. The argument is how we should interface this functionality. NSM is trying to change the way networking is done. We are inspired by the cloud-native principles, we want to target true cloud-native, declarative APIs. What I see in the device plugin framework (kubelet implementation etc) is more suitable to what CNI and Multus do. That is fine, but we need something different. If this means we should challenge the implementation of the existing components- so be it. Otherwise, we are reimplementing the same model and call it different.

Copying what was discussed in #nsm-dev on Slack, for tracking the discussion:

So after a bit of discussing more conceptually, the way that we approach this is kind of two fold:
 * the SRIOV forwarder will be providing connectivity
 * we build higher level Network service concepts on top of it. They expand our current way of declaring services by extending the Mechanism API

As was pointed out several times here - DP is the best way to go further today. That is the path we will follow.

I will be putting a couple of remarks in the spec where I have concerns, but these are more clarifications.

@przemeklal my major ask would be that we limit the impact of the PR on components that are not part of the SRIOV forwarder. That can happen in a follow-up PR.
Overall I am fine with the approach as we have a better story of how to build fast networking with NSM on top of SRIOV now.

Agree to make the impact as low as possible (preferably none) and I hope the ongoing refactoring work should make that easier.

I'd like to invite everyone interested to comment here: networkservicemesh/api#15

This issue has been automatically marked as stale because it has not had activity in 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.