Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAC Vendor is not recognised for BC:24:11 #618

Closed
2 tasks done
n1nj4888 opened this issue Apr 6, 2024 · 21 comments
Closed
2 tasks done

MAC Vendor is not recognised for BC:24:11 #618

n1nj4888 opened this issue Apr 6, 2024 · 21 comments
Labels
bug 🐛 Something isn't working next release/in dev image🚀 This is coming in the next release or was already released if the issue is Closed.

Comments

@n1nj4888
Copy link

n1nj4888 commented Apr 6, 2024

Is there an existing issue for this?

Current Behavior

The MAC Vendor is not recognised for BC:24:11 prefixed mac addresses. They show in devices with a mac vendor of "(Unknown)" ...

Expected Behavior

The MAC Vendor for BC:24:11 prefixed mac addresses should be shown as "Proxmox Server Solutions GmbH" according to https://macvendors.com

Steps To Reproduce

Create a device with a virtual mac address of BC:24:11:xx:xx:xx and it will show as being of vendor "(Unknown)" rather than "Proxmox Server Solutions GmbH" in pialert.

pialert.conf

No response

docker-compose.yml

No response

What branch are you running?

Dev

pialert.log

No response

Debug enabled

  • I have read and followed the steps in the wiki link above and provided the required debug logs and the log section covers the time when the issue occurs.
@n1nj4888 n1nj4888 added the bug 🐛 Something isn't working label Apr 6, 2024
@BambamNZ
Copy link

Could we add 84:28:59, c4:82:e1 & D8:D6:68 to the list as well?
Belongs to Amazon & Tuya
https://maclookup.app/search/result?mac=84:28:59:a7:dc:ad - > Amazon
https://maclookup.app/search/result?mac=c4:82:e1:33:39:c9 -> Tuya
https://maclookup.app/search/result?mac=d8:d6:68:dd:28:6b -> Tuya

Possibilty the option to do lookups via https://macaddress.io/ api to update local Unkown OUI's ?

@fcschow
Copy link

fcschow commented May 1, 2024

Some observations pertain to the vendor update workflow.

  1. back/update_vendors.sh fails

/usr/share/ieee-data path does not exist (at least for my Proxmox LXC bookworm pve).

back/update_vendors.sh
...
echo Updating... /usr/share/ieee-data/
cd /usr/share/ieee-data/ || { echo "could not enter /usr/share/ieee-data directory"; exit 1; }
...

Also, ieee-oui.txt does not seem to have not updated during installation

  1. dev_MAC in database vs Vendor MAC in ieee-oui.txt

dev_MAC is stored in lower case Vendor MAC is upper case. Two never matches during update_vendors workflow.

server/device.py
...
for line in f:
if line.startswith(mac_start_string6):
parts = line.split(' ', 1)
if len(parts) > 1:
...

  1. ieee-oui.txt colunm separator is tab not space

First vendor name will be missing.

server/device.py
...
parts = line.split(' ', 1)
if len(parts) > 1:
vendor = parts[1].strip()
...

  1. (Unknown) vs (unknown)

The sql query is looking for '(unknown)'. Arp-scan returns '(Unknown)'.

front/plugins/vendor_update/script.py
...
FROM Devices
WHERE dev_Vendor = '(unknown)'
OR dev_Vendor = ''
OR dev_Vendor IS NULL
""")
...

Other possible patterns include

(Unknown) (802.1Q VLAN={vlanID})
(Unknown: locally administered) (802.1Q VLAN={vlanID})

  1. Workaround

I ended up doing the following.

  1. Manually replaces ieee-oui.txt in the container at /usr/share/arp-scan/ieee-oui.txt
  2. Deletes the offended device
  3. Allows arp-scan workflow to recover the deleted device

@jokob-sk
Copy link
Owner

jokob-sk commented May 1, 2024

@fcschow thanks a lot for this investigation - this helps a lot to quickly try to fix things.

  1. I think this is the root of the issue - I think the vendors update is not working properly since possibly the alpine rebase - @vladaurosh what do you think can be done here? I quickly checked and /ieee-data directories referenced in this script seem to be missing in the container. Could it be that we are missing a package?
  2. I implemented case-insensitive matching, just to be double-sure this is not the issue. This is available in the latest netalertx-dev image.
  3. I don't think this is the issue as vendors on my always fresh dev instance get recognised just fine

First vendor name will be missing.

server/device.py
...
parts = line.split(' ', 1)
if len(parts) > 1:
vendor = parts[1].strip()
...

I don't think that's the case, as the script cycles thru all lines in that file and matches the beginning of the line with the beginning of the device mac:

# Open the vendors file
with open(vendorsPath, 'r') as f:
    # Iterate over each line in the file
    for line in f:
        # Convert the line to lowercase for case-insensitive matching
        line_lower = line.lower()
        
        # Check if the lowercase line starts with the MAC address prefix
        if line_lower.startswith(mac_start_string6):                 
            # Split the line into two parts using space as the delimiter
            parts = line.split(' ', 1)
            # Check if there are at least two parts after splitting
            if len(parts) > 1:
                # Extract the vendor from the second part and strip whitespace
                vendor = parts[1].strip()
                # Log debug information
                mylog('debug', [f"[Vendor Check] Found '{vendor}' for '{pMAC}' in {vendorsPath}"])
                # Return the vendor
                return vendor
            else:
                # Log an error if the line couldn't be processed after matching
                mylog('debug', [f'[Vendor Check] ⚠ ERROR: Match found, but line could not be processed: "{line_lower}"'])
                # Return -1 to indicate error
                return -1
  1. In SQLite, by default, comparisons such as string matching are not case-sensitive. This means that searching for "(unknown)" or an empty string ('') would match regardless of the case.

@jokob-sk
Copy link
Owner

jokob-sk commented May 2, 2024

I think it's a missing perl package - trying to rebuild it with it and see if that helps.

jokob-sk pushed a commit that referenced this issue May 2, 2024
@jokob-sk
Copy link
Owner

jokob-sk commented May 2, 2024

This should be fixed in the next release. If you can, please have a look at the netalertx-dev docker image, in about 15 minutes (or after the last action finishes) from now.

It would be great if you could test this (backup everything first or use a new container) on your end by switching to the above image and letting me know if the issue was resolved.

Thanks in advance,
j

@jokob-sk jokob-sk added the next release/in dev image🚀 This is coming in the next release or was already released if the issue is Closed. label May 2, 2024
@fcschow
Copy link

fcschow commented May 2, 2024

I misspoke on the 'First vendor name missing'. It should be that the first part of the vendor's name missing. For example,
'Proxmox Server Solutions GmbH' shows up as 'Server Solutions GmbH'.

See ieee-oui.txt file structure.

@vladaurosh
Copy link
Contributor

Hey @jokob-sk

You are right, get-iab and get-oui are perl scripts. Perl will increase image size for around 40MB. I was looking into this last night, I might come up with solution that doesn't use perl.

jokob-sk pushed a commit that referenced this issue May 2, 2024
@jokob-sk
Copy link
Owner

jokob-sk commented May 2, 2024

Hi @vladaurosh - thanks for looking into it - I think I solved it with the last 2 commits without using Perl
@fcschow - please try the latest dev build if you still face these issues - you will have to wait until the vendors update plugin runs, or run it manually

@vladaurosh
Copy link
Contributor

That's awesome @jokob-sk

@fcschow
Copy link

fcschow commented May 2, 2024

Thanks for the quick turnaround..

Please note vendor name will be stored in lower case as follows.
...
line_lower = line.lower() # Convert line to lowercase for case-insensitive matching
if line_lower.startswith(mac_start_string6):
parts = line.split(' ', 1)
...
May I recommend
...
mac_start_string6 = mac_start_string6.upper()
...
for line in f:
...
if line.startswith(mac_start_string6):
...

@jokob-sk
Copy link
Owner

jokob-sk commented May 2, 2024

Hi @fcschow ,

Thanks for checking, but the vendor name is not stored in lower case. I checked it in many setup and also the code only does the comparison on the lower case line version, the actual line variable is used to extract the vendor:

            for line in f:
                line_lower = line.lower()  # Convert line to lowercase for case-insensitive matching
                if line_lower.startswith(mac_start_string6):                 
                    parts = line.split(' ', 1) # 👈 line variable used not line_lower
                    if len(parts) > 1:
                        vendor = parts[1].strip()
                        mylog('debug', [f"[Vendor Check] Found '{vendor}' for '{pMAC}' in {vendorsPath}"])
                        return vendor
                    else:
                        mylog('debug', [f'[Vendor Check] ⚠ ERROR: Match found, but line could not be processed: "{line_lower}"'])
                        return -1

@jokob-sk
Copy link
Owner

jokob-sk commented May 2, 2024

Hi @fcschow ,

I misspoke on the 'First vendor name missing'. It should be that the first part of the vendor's name missing. For example, 'Proxmox Server Solutions GmbH' shows up as 'Server Solutions GmbH'.

See ieee-oui.txt file structure.

I also couldn't reproduce this behavior. Could you please provide me with screenshots?

@fcschow
Copy link

fcschow commented May 3, 2024

Sorry, my bad. U r correct.

Hi @fcschow ,

Thanks for checking, but the vendor name is not stored in lower case. I checked it in many setup and also the code only does the comparison on the lower case line version, the actual line variable is used to extract the vendor:

            for line in f:
                line_lower = line.lower()  # Convert line to lowercase for case-insensitive matching
                if line_lower.startswith(mac_start_string6):                 
                    parts = line.split(' ', 1) # 👈 line variable used not line_lower
                    if len(parts) > 1:
                        vendor = parts[1].strip()
                        mylog('debug', [f"[Vendor Check] Found '{vendor}' for '{pMAC}' in {vendorsPath}"])
                        return vendor
                    else:
                        mylog('debug', [f'[Vendor Check] ⚠ ERROR: Match found, but line could not be processed: "{line_lower}"'])
                        return -1

@fcschow
Copy link

fcschow commented May 3, 2024

My bad. I see it now. The code uses the custom ieee-oui_all_filtered.txt file for looking up the vendor MAC/name. My apologies.

Hi @fcschow ,

I misspoke on the 'First vendor name missing'. It should be that the first part of the vendor's name missing. For example, 'Proxmox Server Solutions GmbH' shows up as 'Server Solutions GmbH'.
See ieee-oui.txt file structure.

I also couldn't reproduce this behavior. Could you please provide me with screenshots?

@jokob-sk
Copy link
Owner

jokob-sk commented May 3, 2024

All good @fcschow - it's always better to have someone check the code for bugs :)

@fcschow
Copy link

fcschow commented May 3, 2024

Hi @jokob-sk,

I am having issue with netalertx-dev:latest and netalertx-dev:sha-45be8a0 from dockerHub. The settings page is not rendering properly See below.

image

image

image

In essence, all plugins configurations are not rendered.

image

If I revert back to netalertx:latest. The page renders properly.

This should be fixed in the next release. If you can, please have a look at the netalertx-dev docker image, in about 15 minutes (or after the last action finishes) from now.

It would be great if you could test this (backup everything first or use a new container) on your end by switching to the above image and letting me know if the issue was resolved.

Thanks in advance, j

@jokob-sk
Copy link
Owner

jokob-sk commented May 3, 2024

Hi @fcschow ,

Can you please post a screenshot of the browser dev console of any errors that are logged?

I think refreshing the cache/using an incognito window or clicking the refresh button might fix this:

image

@fcschow
Copy link

fcschow commented May 4, 2024

Brilliant. Runs in incognito window fixed the issue. Thanks @jokob-sk.

@fcschow
Copy link

fcschow commented May 6, 2024

Hi @jokob-sk ,

Observation again. Plz see /app/server/device.py below.

filePath = vendorsPath

if os.path.isfile(vendorsPathNewest):
    filePath = vendorsPathNewest 
...
try:
    with open(vendorsPath, 'r') as f:      <----  should it be filePath  ??
        for line in f:

jokob-sk pushed a commit that referenced this issue May 6, 2024
@jokob-sk
Copy link
Owner

jokob-sk commented May 6, 2024

@fcschow - you are correct - thanks! fixed :)

@jokob-sk
Copy link
Owner

jokob-sk commented May 9, 2024

released -> closing for now

@jokob-sk jokob-sk closed this as completed May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working next release/in dev image🚀 This is coming in the next release or was already released if the issue is Closed.
Projects
None yet
Development

No branches or pull requests

5 participants