Python 3.12 is hitting Fedora Rawhide now, so this would be nice to get address soon if possible.

Thanks Orion.
Looks like it's ultimately the time to say goodbye to 2.7.
I'll convert the codebase of master (1.0) to 3.x, remove old/compat code pieces and perform the necessary switch to asyncio. Since this is a major change, it becomes v.1.1 from that point.
Versions 1.0/0.11/0.10 (and old 0.9) would remain 2.x based for people that still need them for old python versions (the config is compatible yet, so for a while the filters can be shared across all branches).

Related #2690

So the migration to 3.x is fulfilled (#2690 is closed) now...
Lets try to switch from asyncore to asyncio...
Just it would be not so easy, since event loop of asyncio is not really suitable for usage together with threads (which is currently a basis of fail2ban). One could write some mixed code (integrate to the life span, etc), just it looks ugly, let alone coroutines are not intended for parallel execution in threads and the chain asyncio <-> CORO <-> Thread <-> threaded code would expects some interlayers and mockups at multiple cross points.
Better would by to switch from threading to asyncio completely (it would have also additional pros, like no GIL, fully asynchronous handling in all modules, etc), but it implies rewrite of everything, and the effort is enormous.

Where do I got the version ported to 3.12? I just grabbed from github and it is still failing with failure to find module asynchat.

Where do I got the version ported to 3.12

Nowhere at the moment. As you can see the issue is still open.

My apologies, I should wake-up before I attempt to read this stuff.
Last failed login: Sun Jul 9 03:15:01 PDT 2023 from 180.101.88.248 on ssh:notty
There were 6132 failed login attempts since the last successful login.

Since this is still preventing me from building in Fedora Rawhide I thought I'd ask. Is the module used for all tests or just some? If only some, I could disable those tests?

Is the module used for all tests or just some?

Nope! This is one of main modules of fil2ban-server (also used for communication between client and server).
We definitely need to rewrite it (or find a suitable replacement).

Would it make sense to (at least temporarily) use pyasynchat and pyasyncore on Python 3.12+? The imports should remain compatible, so it boils down to declaring a dependency on them with an environment marker, e.g.:

pyasyncore;python_version>="3.12"
pyasynchat;python_version>="3.12"

Note that even with that resolved there's another issue - tests/action_d/test_smtp.py uses smtpd, which is also removed from Python 3.12. The recommended replacement is aiosmtpd, but it's definitely not just a drop-in replacement.

As for test_smtp.py it is not a large issue, because we could disable it for 3.12 as known issue, and the action. which is only dynamically loaded, is unusable then, until becomes rewritten (there are enough another mailing actions and they are optional and even not really recommended).

Regarding pyasynchat and pyasyncore, I'm still unsure:

• are there always available?
• don't they get a mentioned problematic related to asyncio and threads... must try it.

I don't think the smtpd removal affects any 'real' code - it's just the test case that uses it, as a way to check the smtp actions are doing what they should. So I think the actions should keep working fine (unless something breaks them and we don't notice because the test doesn't work :>), it's just the test case that can't work.

This test covers action.d/smtp.py - I thought it may be affected similarly (didn't test it yet with 3.12), but if not, the action would be untested...

However another worse thing would be - disabling that test will decrease coverage for usage of pythonic actions under real circumstances (it was sole stock pythonic action tested completely), so we'd need either another pythonic (dummy-)action or another SMTP server for test.

ah, yeah, that sucks. but yes, theoretically at least, the action should still work. it doesn't use smtpd, because smtpd is a server library, and the action only wants to act as a client. It uses client-y things - smtplib, socket, email.(stuff) - and those have not been removed.

It's just the test case that uses the server-y smtpd, because it needs to act like a server to test that the action is being a good client.

I installed pyasynchat from https://github.com/simonrob/pyasynchat and pyasyncore from https://github.com/simonrob/pyasyncore . This is a temporary method until a fail2ban will be rebased on asyncio (https://docs.python.org/3/library/asyncio.html). I don't use smtpd module. fail2ban works for me with python 3.12 on Gentoo linux fine.

OK, there is new branch python-3.12--asyncore that actually works with 3.12 in CI of GHA. But:

• asyncore/asynchat will be installed via pip: python -m pip install pyasynchat and python -m pip install pyasyncore
• test_smpt is completely ignored
• testSetupInstallRoot fails, because of setuptools issue (probably because of missing distutils). No idea what is wrong here, either it is the issue on that Ubuntu box only, or setuptools is an issue with 3.12 basically.

The issue with setuptools is fixed now (it should be installed for python version used to test, whereas apt seems to install it for stock python version). Anyway it works now.

This can be closed (and get a TODO "rewrite f2b to asyncio"), but...

@fail2ban/maintainers, @opoplawski, @sylvestre what should we do with asynchat and asyncore now?

Please vote in following 2 comments, 👍 (in favor), 👎 (against) and 👀 (abstain).
The choice 👍 for both variants means it'd try to import pyasynchat/pyasyncore and if missed as fallback would use local asynchat/asyncore modules.

Variant I:

we will use pyasynchat and pyasyncore packages provided by pip, distro packet-manager, etc

Variant II:

we shall add copies of asynchat and asyncore as local modules of fail2ban (libraries get bundled in fail2ban)

Hopefully I interpreted things correctly but in the case of Fedora, internet access is not allowed during the build process. We would just need the two libraries bundled in fail2ban. Additionally we're almost to the point of having Python 3.13 in Rawhide and I'm not sure if that causes any additional issues or not...

internet access is not allowed during the build process

Well, one doesn't need them during build process. Variant I means that fail2ban package should get a dependency to pip- or distro- or whatever else package. No idea what exactly is possible and which packages are available in distros (I'm not maintainer).

we would just need the two libraries bundled in fail2ban.

Then you'd vote for Variant II.

Additionally we're almost to the point of having Python 3.13 in Rawhide and I'm not sure if that causes any additional issues or not...

Yes it caused few compat issues (deprecated removals only)...
I fixed it in other branch, see https://github.com/fail2ban/fail2ban/actions/runs/7182148498/job/19558003605
Would merge it to master soon.

awesome, thanks a lot!

Hope this makes it inside 24.04 Soon

wiki :: Manual installation of debian upstream-package released here...
I assume, it'd be compatible to ubuntu 24.

We do server management and use tools like Ansible. Doing manual installation on 1000's of servers is more of a headache and needs custom scripts to keep it updated. So hope Ubuntu finds this asap and uses a later version

Thanks for help and prompt response

Im still getting No module named 'asynchat' when trying 1.1.0-1 upstream-package. What am i doing wrong? Running Ubuntu 24.04-rpi5.

Edit: I tried installing pyasynchat, didn't solve it

Im still getting No module named 'asynchat' when trying 1.1.0-1 upstream-package. What am i doing wrong? Running Ubuntu 24.04-rpi5.

Edit: I tried installing pyasynchat, didn't solve it

I'm not sure how, but i think i solved it by uninstalling f2b, re-installing 1.0.2 via apt, stopping the service and client, finally installing 1.1.0-1 over it.

I can only recommend everybody who is affected to vote the bugreport
https://bugs.launchpad.net/ubuntu/+source/fail2ban/+bug/2055114
by logging in and clicking the green text below the headline. You can mark yourself as affected by this.
Maybe the bug will gain enough attention to be fixed sooner.

Here is how to fix the problem

wget https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1/+build/28291332/+files/fail2ban_1.1.0-1_all.deb
sudo dpkg -i fail2ban_1.1.0-1_all.deb

systemctl enable fail2ban
service fail2ban start
fail2ban-client status

Here is how to fix the problem

wget https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1/+build/28291332/+files/fail2ban_1.1.0-1_all.deb
sudo dpkg -i fail2ban_1.1.0-1_all.deb

systemctl enable fail2ban
service fail2ban start
fail2ban-client status

It really depends, whether whether this fixes the issue or not. At least for me, as a user of Plesk, some jails won't work anymore. Instead, installing the pyasynchat extension with pip actually fixes the issue. At least Plesk's fail2ban integration keeps working which is broken by installing the newer version from launchpad.

At this point it sounds like maybe there's a downstream issue - Ubuntu and/or Debian maybe did not introduce a package dependency on the split-out asynchat/asyncore libraries when they packaged the new release. It might be better to report that to Debian/Ubuntu package maintainers? At least looking at https://packages.debian.org/sid/fail2ban , it doesn't look like any dependency on asynchat/asyncore is listed there. @sylvestre seems to be the Debian maintainer - did you miss this new dep, or am I misreading the Debian metadata?

Will the fix that comes along be able to solve our problems?

Or, will it require a reinstall?

Well, there may be still an issue with asyncore: GHA shows that pyinotify may need it, so f2b may fail to initialize pyinotify backend.

Possible solution:

• newer version of pyinotify python module without a dependency to asyncore
• the fix 054e1d8 (fallback to local libraries if import miss them) must be extended, e. g. extending a runtime PYTHONPATH to compat folder

Related to CI it seems to be alright now (so 304c3cd ultimately closes this).

Here is how to fix the problem

wget https://launchpad.net/ubuntu/+source/fail2ban/1.1.0-1/+build/28291332/+files/fail2ban_1.1.0-1_all.deb
sudo dpkg -i fail2ban_1.1.0-1_all.deb

systemctl enable fail2ban
service fail2ban start
fail2ban-client status

It Worked in Ubuntu 24.x. THanks