The timestamps of the logs are in this format : "[21:20:18]" (no date or year), so I'm not sure what value would "time" in failJSON have.

The time simulating by test suite for NOW is 1124013600 - Sun, 14 Aug 12:00:00 CEST 2005 (in the time zone CET).
But because [21:20:18] lies in the future, fail2ban would assume failure from previous day, so it results to 2005-08-13T21:20:18.

Coverage remained the same at 98.104% when pulling 778b094 on MinePro120:master into 960e30c on fail2ban:master.

Codecov Report

Merging #2852 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2852   +/-   ##
=======================================
  Coverage   95.91%   95.91%           
=======================================
  Files          79       79           
  Lines       15978    15978           
  Branches     2615     2615           
=======================================
  Hits        15326    15326           
  Misses        344      344           
  Partials      308      308           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 960e30c...43a9ce9. Read the comment docs.

Codecov Report

Merging #2852 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2852   +/-   ##
=======================================
  Coverage   95.91%   95.91%           
=======================================
  Files          79       79           
  Lines       15978    15978           
  Branches     2615     2615           
=======================================
  Hits        15326    15326           
  Misses        344      344           
  Partials      308      308           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 960e30c...9d16444. Read the comment docs.

Any news about this pull request?

I don't think it is justified to merge this PR, because it still looks "vulnerable" at the moment.
Moreover, take a closer look to testcase - it looks like a single attempt will be counted 3 times (as it'd be 3 attempts), what is not correct in my opinion.

If the log-file containing the message like this:

[minecraft]
port = ...
logpath = ...
filter = 
failregex = ^[^:]*: Disconnecting \S+ \(/<ADDR>:\d+\): You are not white-listed on this server!
enabled = true

This is still a non-working solution, but it builds on @sebres suggestion:

failregex = \[(?P<time>\d+:\d+:\d+)\].*\(\/(?P<host>\d+.\d+.\d+.\d+):\d+\) lost connection.*$

The host and time capture attempt to comply with some fail2ban specifications, but Minecraft Servers output the timestamp in a format that seems incompatible with the fail2ban time regex. Which I think is what @MinePro120 and @sebres were discussing earlier. I'm looking for a way to modify how the Minecraft server reports the date and time in the logs. Still, all I've found so far are guides for tweaking log4j; from my understanding, that logger was deprecated for vulnerabilities.

This is still a non-working solution

Why then do you provide such non-working "solution" here?

I'm looking for a way to modify how the Minecraft server reports the date and time in the logs.

You don't need... One can simply specify datepattern for the jail or filter.
It looks like you'd need:

datepattern = ^\[%%H:%%M:%%S\]\s+

By the way - usage of catch-all's is ugly and .*$ is not an anchor at all, so either remove it completely (if it is justified unanchored) or rewrite catch-all more precisely.
And don't use captured groups of (?P<host>...), use <ADDR> (or <HOST>) instead.

Hey, thanks for the reply.

Why then do you provide such non-working "solution" here?

Because this is a collaborative platform, and instead of being snarky, I'll contribute my limited knowledge so that others may improve upon it.

You don't need... One can simply specify datepattern for the jail or filter.

The date patterns the logs provide resolve from seconds up to the hour, so there is no day, month, or year information. Thanks for that link to the documentation; I see that by specifying datepattern as NONE we could bypass that limitation.

I don't know what the fial2ban best practices are. Would you recommend keeping the datepattern matching the time from seconds up to the hour or setting it to NONE?

Regarding the use of catch-alls not sure how to avoid using it since the log entries are quite heterogeneous. It might not be the best solution, but if it works, it provides something for others with the skills to improve upon it. I was testing it against the regex 101 Python option, and .*$ seemed to work; I might not be understanding your comment correctly.

And don't use captured groups of (?P...), use (or ) instead.

Just followed what the DOCs I had available said.

https://www.fail2ban.org/wiki/index.php/MANUAL_0_8#:~:text=for%20%22permanent%22%20ban.-,Filters,-The%20directory%20filter

Hope this helps anyone trying to improve on this solution, it would be really cool to have something like this working (even if it's just remedially) to get rid of the robot-spam on the server.

This isn't quite the same as the use case mentioned in the aforementioned PR, but I've found using a systemd service for logging means fail2ban can utilize systemd-journal's timestamps, and a regex that matches the whitelist errors works. That's not exactly the bot problem this discussion has been about, but it's better than nothing.

Here's my minecraft-whitelist.conf:

[INCLUDES]
before = common.conf

[Definition]
failregex = Disconnecting .* \(/<ADDR>:.*\): You are not white-listed on this server!
journalmatch = _SYSTEMD_UNIT=$SERVICE.service + _COMM=$START.sh

where $SERVICE is the name of the systemd service and $START.sh is the script or binary that starts the server (usually ServerStart.sh or something similar).

I'm also aware there aren't any anchors, I've had some trouble with them in the past for whatever reason.

I'm also aware there aren't any anchors, I've had some trouble with them in the past for whatever reason.

Well it depends on log-format (and log-target, e. g. systemd journal or log-file), here is fully-anchored example for log-line from this PR:

failregex = ^[^:]*: Disconnecting [^\(]*\(/<ADDR>:\d+\): You are not white-listed on this server!

([^:]* and [^\(]* are not catch-alls and mean everything before colon and open round parenthesis, respectively)

Alternatively it can be only anchored from the end, with few conditions:

• there should be no catch-alls between <ADDR> and determining message (exactly the case here);
• logging of free message content (some foreign user input) at the end of the log is explicitly excluded (hardly possible to estimate without to review the source code or asking developers).

failregex = \(/<ADDR>:\d+\): You are not white-listed on this server!\s*$

I think the issues I ran into were related to extraneous timestamps at the beginning of a log entry, despite using systemd-journal. Basically, the timestamp mentioned earlier in this thread is still present in the journal entry, so when I tried to anchor at the beginning, I'm guessing my regex was having trouble with that initial timestamp when it had already successfully parsed the journal's timestamp. I don't know if fail2ban tries to parse timestamps again after successfully getting the timestamp from the journal or if I need to add something to the regex to make it ignore that initial section in square brackets with the timestamp. I can't tell if that's what your regex does, it doesn't look like it does.

The other issue I may have been facing was when I heard anchoring, I thought it meant on both sides (both beginning and end). Is it preferred to only anchor from one or the other and not both?

I think in this specific case for minecraft-whitelist, anchoring from the end would work just fine, I don't think there is any added input in between.

I thought it meant on both sides (both beginning and end).

It is true, either if one needs to be precise as possible, or if there are some catch-alls similar REs used inside, before the captured data in left anchored RE, or after captured data in right anchored RE (quasi to restrict corner cases for injection attempts via foreign data).

Is it preferred to only anchor from one or the other and not both?

It depends, but generally spoken - no, it is not preferred.
If there is no catch-alls and matched message part is obvious a failure and such match allows unambiguously distinguish failed messages from the rest, one can surely use the anchor from one side only (mostly begin-anchor).
This has also another pros, like it wouldn't matter for which reason the failure happens, it is still a failure and it'd be considered as a failed attempt regardless what devs of service would devise later (or which info the log may get latter additionally).

For instance an RE ^\s*authentication failed for user "[^"]+": would match all that:

authentication failed for user "usr1": user unknown
authentication failed for user "usr2": password mismatch
authentication failed for user "usr3": auth-service unavailable
authentication failed for user "usr4": unsupported method
authentication failed for user "usr5": whatever reason devs invent tomorrow or more data gets included later

And if something of that shall not cause a failure (e. g. 3rd message), it can be pretty simple done with negative lookahead: ^\s*authentication failed for user "[^"]+": (?!auth-service unavailable)

It is hardly possible to give an unique answer for general usecase.

I think in this specific case for minecraft-whitelist, anchoring from the end would work just fine, I don't think there is any added input in between.

"I don't think" is not equal "I know", what is mostly expected for a stock filter.
You can basically do what you want on your hosts (since it is your responsibility).
But in case of some OSS-project, we must ensure it'd not cause false positives (or be affected by some inject-attempts).