Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Active connection tracking in ebpf #32584

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

AwesomePatrol
Copy link
Contributor

@AwesomePatrol AwesomePatrol commented May 16, 2024

As described in cilium/design-cfps#31, this is part of #31752

Example configuration (in cilium-config) for KIND cluster:

  fixed-zone-mapping: zone-a=123,zone-b=129
  enable-active-connection-tracking: "true"

where topology labels were set on the nodes:

kubectl label node kind-control-plane topology.kubernetes.io/zone=zone-a
kubectl label node kind-worker topology.kubernetes.io/zone=zone-b

Zone mapping can be verified with cilium map get cilium_lb4_backends_v3:

Key   Value                        State   Error
16    ANY://10.244.1.74[zone-b]    sync
17    ANY://10.244.1.74[zone-b]    sync
18    ANY://10.244.1.202[zone-b]   sync
19    ANY://10.244.1.202[zone-b]   sync
13    ANY://192.168.11.2[zone-b]   sync
14    ANY://10.244.0.178[zone-a]   sync
15    ANY://10.244.0.178[zone-a]   sync

LRS accounting can be verified with cilium map get cilium_lb_act:

Key                       Value     State   Error
10.96.138.80:80[zone-b]   +72 -72
10.96.138.80:80[zone-a]   +28 -28
Add cilium_lb_act BPF map with counters of opened and closed connections

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label May 16, 2024
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from e8f0244 to beecf87 Compare May 16, 2024 14:13
@maintainer-s-little-helper

This comment was marked as resolved.

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label May 16, 2024
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from c7a9677 to 58f239e Compare May 16, 2024 17:02
@maintainer-s-little-helper

This comment was marked as resolved.

@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from 58f239e to 6bd2077 Compare May 17, 2024 07:03
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-sign-off The author needs to add signoff to their commits before merge. label May 17, 2024
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from 6bd2077 to 66e1d80 Compare May 17, 2024 07:12
@AwesomePatrol

This comment was marked as outdated.

@AwesomePatrol AwesomePatrol marked this pull request as ready for review May 17, 2024 08:08
@AwesomePatrol AwesomePatrol requested review from a team as code owners May 17, 2024 08:08
@AwesomePatrol

This comment was marked as outdated.

Copy link
Contributor

@lmb lmb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loader changes are fine. As a rando reviewer, it's not at all clear to me what lrs stands for however.

@tommyp1ckles tommyp1ckles requested review from julianwiedmann and removed request for danehans May 20, 2024 04:20
@tommyp1ckles
Copy link
Contributor

@julianwiedmann would you be able to review this for @cilium/sig-datapath, I'm assigned sig-cli but my review would also count towards datapath - and I'm not familiar enough with this code to provide a thorough review.

@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from 76f4e16 to 27af564 Compare May 21, 2024 16:25
@AwesomePatrol
Copy link
Contributor Author

Loader changes are fine. As a rando reviewer, it's not at all clear to me what lrs stands for however.

LRS stands for Load Reporting Service from Envoy. Please see: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/load_stats/v3/lrs.proto. I am bad at naming stuff, so I am open to suggestions

@tommyp1ckles
Copy link
Contributor

@AwesomePatrol Does the TODO list in the second commits message represent work that's still to be done?

pkg/maps/lrs/lrs.go Outdated Show resolved Hide resolved
pkg/option/config.go Outdated Show resolved Hide resolved
@julianwiedmann
Copy link
Member

Based on the TODOs and the missing description in the second patch, this looks like it still needs some more work. Flipping to draft for now to get it off my queue, please flip back when you're ready! 🙏

@julianwiedmann julianwiedmann marked this pull request as draft May 22, 2024 10:32
@pchaigno pchaigno added the sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. label May 23, 2024
@pchaigno pchaigno added release-note/major This PR introduces major new functionality to Cilium. feature/conntrack labels May 23, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels May 23, 2024
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from 27af564 to b97e055 Compare May 23, 2024 10:40
@AwesomePatrol
Copy link
Contributor Author

AwesomePatrol commented May 23, 2024

@AwesomePatrol Does the TODO list in the second commits message represent work that's still to be done?

Yes and no. TODOs describe work that will be done in PR that starts exposing these counters as Prometheus metrics (not this one).

Based on the TODOs and the missing description in the second patch, this looks like it still needs some more work. Flipping to draft for now to get it off my queue, please flip back when you're ready! 🙏

I removed TODOs from the commit message as to not confuse any other reviewers. What descriptions are missing?

I migrated to cell/hive and renamed lrs to act (standing for Active Connection Tracking) in order to avoid confusion with xDS workstream (related, but not the same).

@AwesomePatrol AwesomePatrol marked this pull request as ready for review May 23, 2024 10:43
@AwesomePatrol AwesomePatrol marked this pull request as draft May 23, 2024 10:45
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch 2 times, most recently from 323a58c to 36a789c Compare May 23, 2024 12:08
@AwesomePatrol

This comment was marked as outdated.

@pchaigno
Copy link
Member

I removed TODOs from the commit message as to not confuse any other reviewers. What descriptions are missing?

I'm guessing Julian means the commit descriptions. They should describe what the commit is doing, but more importantly why and any subtleties to help reviewers understand what is going on. See examples in the bpf/ commit history. Of course, depending on the changes in the commit, that can be very short (if what and why are obvious with no subtleties). With hundreds of LoC, I'm going to venture your changes aren't trivial enough to have empty commits 😁

@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from 36a789c to 4571040 Compare May 27, 2024 06:06
@AwesomePatrol
Copy link
Contributor Author

/test

@AwesomePatrol AwesomePatrol marked this pull request as ready for review May 27, 2024 07:21
pkg/maps/act/act.go Outdated Show resolved Hide resolved
pkg/maps/act/act.go Outdated Show resolved Hide resolved
pkg/maps/act/act.go Outdated Show resolved Hide resolved
Copy link
Contributor

@joamaki joamaki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving as I'm going on vacation and mostly had nits about unused methods.

This is needed to break a cyclic dependency on LB{4,6}_map from lb.h in
lrs.h (next patch) imported by conntrack.h.

Signed-off-by: Aleksander Mistewicz <amistewicz@google.com>
Add new map - LB_ACT_MAP - behind ENABLE_ACTIVE_CONNECTION_TRACKING flag
with counters of opened and closed connections. Behavior of eBPF remains
completely unchanged when ENABLE_ACTIVE_CONNECTION_TRACKING flag is not
set.

When an entry, to conntrack table (with dir == CT_SERVICE) is created,
an entry in LB_ACT_MAP.opened is incremented by one. The same occurs
when a connection is reopened. When connection is closed (action ==
ACTION_CLOSE and dir == CT_SERVICE), the related LB_ACT_MAP.closed is
incremented by one.

LB_ACT_MAP is keyed by svc_id (also known as rev_nat_index) and zone.
Backend ID is used to fetch the backend definition from
LB{4,6}_BACKEND_MAP which also contains zone field. This field is
populated only when EndpointSlice contains a reference to zone in
FixedZoneMapping (so it is possible to convert between uint8 ID and
string).

Signed-off-by: Aleksander Mistewicz <amistewicz@google.com>
Signed-off-by: Aleksander Mistewicz <amistewicz@google.com>
Signed-off-by: Aleksander Mistewicz <amistewicz@google.com>
Set ACT map size to backends*zones

Remove unnecessary methods for ACT key/value

Remove ACT key/value's deepcopy code as it is not needed

Signed-off-by: Aleksander Mistewicz <amistewicz@google.com>
@AwesomePatrol AwesomePatrol force-pushed the active-connection-tracking-in-ebpf branch from a0a23e9 to 31d3ec5 Compare May 31, 2024 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/conntrack release-note/major This PR introduces major new functionality to Cilium. sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants